{"@attributes":{"version":"2.0"},"channel":{"title":"Ray Hu","description":"A personal site for technical notes, projects, and selected writing by Ray Hu.","link":"https:\/\/huruilizhen.github.io\/","pubDate":"Sat, 04 Jul 2026 17:25:04 +0000","lastBuildDate":"Sat, 04 Jul 2026 17:25:04 +0000","generator":"Jekyll v3.10.0","item":[{"title":"A Productive C++ Development Environment","description":"<p>In this post, we will explore how to build a productive C++ development environment around <code class=\"language-plaintext highlighter-rouge\">CMake<\/code>, <code class=\"language-plaintext highlighter-rouge\">clangd<\/code>, <code class=\"language-plaintext highlighter-rouge\">lldb<\/code>, and <code class=\"language-plaintext highlighter-rouge\">AstroNvim<\/code>. The focus here is not on collecting more tools, but on making multiple tools share one consistent understanding of the same project so that build, edit, and debug behavior remain predictable.<\/p>\n\n<!--more-->\n\n<h2 id=\"table-of-contents\">Table of Contents<\/h2>\n\n<!--toc:start-->\n<ul>\n  <li><a href=\"#table-of-contents\">Table of Contents<\/a><\/li>\n  <li><a href=\"#why-this-setup-exists\">Why this Setup Exists<\/a><\/li>\n  <li><a href=\"#system-tool-chain-layer\">System Tool Chain Layer<\/a>\n    <ul>\n      <li><a href=\"#compiler\">Compiler<\/a><\/li>\n      <li><a href=\"#build-system\">Build System<\/a><\/li>\n      <li><a href=\"#generators\">Generators<\/a><\/li>\n      <li><a href=\"#compilation-metadata\">Compilation Metadata<\/a><\/li>\n      <li><a href=\"#dependency-strategy\">Dependency Strategy<\/a><\/li>\n      <li><a href=\"#build-type-model\">Build Type Model<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#project-structure-contract\">Project Structure Contract<\/a>\n    <ul>\n      <li><a href=\"#baseline-layout\">Baseline Layout<\/a><\/li>\n      <li><a href=\"#directory-semantics\">Directory Semantics<\/a><\/li>\n      <li><a href=\"#test-directory-organization\">Test Directory Organization<\/a><\/li>\n      <li><a href=\"#constraint-model\">Constraint Model<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#debug-stack\">Debug Stack<\/a>\n    <ul>\n      <li><a href=\"#cli-debugging\">CLI Debugging<\/a><\/li>\n      <li><a href=\"#sanitizers\">Sanitizers<\/a><\/li>\n      <li><a href=\"#ide-debugging\">IDE Debugging<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#integrate-with-astronvim\">Integrate with AstroNvim<\/a>\n    <ul>\n      <li><a href=\"#clangd-integration\">clangd Integration<\/a><\/li>\n      <li><a href=\"#debugger-frontend-integration\">Debugger Frontend Integration<\/a><\/li>\n      <li><a href=\"#community-pack-and-lightweight-configuration\">Community Pack and Lightweight Configuration<\/a><\/li>\n      <li><a href=\"#what-astronvim-actually-needs\">What AstroNvim Actually Needs<\/a>\n<!--toc:end--><\/li>\n    <\/ul>\n  <\/li>\n<\/ul>\n\n<hr \/>\n\n<h1 id=\"why-this-setup-exists\">Why this Setup Exists<\/h1>\n\n<p>The problem with modern C++ development environments is not a lack of tools, but a <strong>lack of consistent contracts<\/strong> between tools. A typical C++ project involves several subsystems that are only loosely coupled with one another: compiler (<a href=\"https:\/\/clang.llvm.org\/\"><code class=\"language-plaintext highlighter-rouge\">clang<\/code><\/a>, <a href=\"https:\/\/gcc.gnu.org\/\"><code class=\"language-plaintext highlighter-rouge\">gcc<\/code><\/a>, <a href=\"https:\/\/visualstudio.microsoft.com\/vs\/features\/cplusplus\/\"><code class=\"language-plaintext highlighter-rouge\">MSVC<\/code><\/a>), build system (<a href=\"https:\/\/cmake.org\/\"><code class=\"language-plaintext highlighter-rouge\">CMake<\/code><\/a>, <a href=\"https:\/\/www.gnu.org\/software\/make\/\"><code class=\"language-plaintext highlighter-rouge\">Make<\/code><\/a>, <a href=\"https:\/\/ninja-build.org\/\"><code class=\"language-plaintext highlighter-rouge\">Ninja<\/code><\/a>), editor (<a href=\"https:\/\/neovim.io\/\"><code class=\"language-plaintext highlighter-rouge\">Neovim<\/code><\/a>, <a href=\"https:\/\/code.visualstudio.com\/\"><code class=\"language-plaintext highlighter-rouge\">VSCode<\/code><\/a>, <a href=\"https:\/\/www.jetbrains.com\/clion\/\"><code class=\"language-plaintext highlighter-rouge\">CLion<\/code><\/a>), language server (<a href=\"https:\/\/clangd.llvm.org\/\"><code class=\"language-plaintext highlighter-rouge\">clangd<\/code><\/a>), debugger (<a href=\"https:\/\/lldb.llvm.org\/\"><code class=\"language-plaintext highlighter-rouge\">lldb<\/code><\/a>, <a href=\"https:\/\/www.gnu.org\/software\/gdb\/\"><code class=\"language-plaintext highlighter-rouge\">gdb<\/code><\/a>), and static analysis tools such as <a href=\"https:\/\/clang.llvm.org\/extra\/clang-tidy\/\"><code class=\"language-plaintext highlighter-rouge\">clang-tidy<\/code><\/a> or sanitizers. The friction does not come from any single tool being weak. It comes from the fact that these tools do not naturally share the same engineering understanding of the project.<\/p>\n\n<p>The core contradiction is that each tool often maintains its own partial model of the same codebase. The build system knows the real include paths, compile flags, macro definitions, and target graph; the language server may not. Without <code class=\"language-plaintext highlighter-rouge\">compile_commands.json<\/code>, <code class=\"language-plaintext highlighter-rouge\">clangd<\/code> is forced to guess, so it is no longer truly understanding the project but only approximating it. This is why code may build correctly while diagnostics, jump-to-definition, or autocomplete still fail inside the editor.<\/p>\n\n<p>The same inconsistency appears between IDE workflows and CLI workflows. A project may compile inside an IDE but fail from the terminal, or the reverse, because one side is silently adding include paths, using a different generator, or building a different configuration such as <code class=\"language-plaintext highlighter-rouge\">Debug<\/code> versus <code class=\"language-plaintext highlighter-rouge\">Release<\/code>. The debugger adds yet another execution layer: after optimization, inlining, and symbol generation, the running program no longer maps perfectly to the source view, which is why breakpoints move, variables disappear, and stack frames sometimes look misleading. In other words, the real problem is not missing tools but <strong>fragmented tool state<\/strong>.<\/p>\n\n<p>The goal of this setup is therefore not convenience in the superficial sense. It is to make the environment reproducible, decoupled, and cognitively simple. Reproducible means that on any machine the workflow should remain <code class=\"language-plaintext highlighter-rouge\">git clone -&gt; configure -&gt; build -&gt; debug<\/code>. Decoupled means the editor does not define the build logic, the language server does not depend on hand-written local flags, and the debugger does not require one specific UI. Simple means the developer only needs to understand how to build, run, and debug, rather than how several tools secretly synchronize with one another.<\/p>\n\n<p>To achieve that, this article treats <code class=\"language-plaintext highlighter-rouge\">CMake<\/code> plus <code class=\"language-plaintext highlighter-rouge\">compile_commands.json<\/code> as the <strong>single source of truth<\/strong>. The desired end state is a CLI-first workflow: you configure the project explicitly,<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>cmake <span class=\"nt\">--preset<\/span> debug\n<\/code><\/pre><\/div><\/div>\n\n<p>build it,<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>cmake <span class=\"nt\">--build<\/span> <span class=\"nt\">--preset<\/span> debug-build\n<\/code><\/pre><\/div><\/div>\n\n<p>and debug the resulting binary directly,<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>lldb .\/path\/to\/your\/binary\n<\/code><\/pre><\/div><\/div>\n\n<p>while the editor remains only a frontend. In this model, <code class=\"language-plaintext highlighter-rouge\">AstroNvim<\/code> reads <code class=\"language-plaintext highlighter-rouge\">compile_commands.json<\/code>, invokes <code class=\"language-plaintext highlighter-rouge\">clangd<\/code>, and attaches to a debug adapter when requested, but it does not participate in the project\u2019s build logic. Here <code class=\"language-plaintext highlighter-rouge\">compile_commands.json<\/code> is not produced by the editor either; it is generated by <code class=\"language-plaintext highlighter-rouge\">CMake<\/code> during the project configuration step and then consumed by other tools as shared build metadata. That separation is the main idea behind the entire setup:<\/p>\n\n<blockquote>\n  <p>C++ development is not IDE-centric. It is build-system-centric.<\/p>\n<\/blockquote>\n\n<p>An IDE can be a very good interface, but it should remain an interpreter of the build system, not the place where the build system is secretly redefined.<\/p>\n\n<hr \/>\n\n<h1 id=\"system-tool-chain-layer\">System Tool Chain Layer<\/h1>\n\n<h2 id=\"compiler\">Compiler<\/h2>\n\n<p>If the previous section explained the motivation, this section defines the <strong>engineering contract<\/strong>. Everything above the compiler and build system, including <code class=\"language-plaintext highlighter-rouge\">clangd<\/code>, the editor, and the debugger, depends on this layer behaving predictably. The invariant is simple: <strong>the build system is the only source of compilation truth<\/strong>.<\/p>\n\n<p>At the compiler layer, the primary compiler in this setup is <code class=\"language-plaintext highlighter-rouge\">clang++<\/code>, while <code class=\"language-plaintext highlighter-rouge\">gcc<\/code> remains a secondary choice for compatibility or distribution-specific environments. On <code class=\"language-plaintext highlighter-rouge\">macOS<\/code>, the practical default is usually <code class=\"language-plaintext highlighter-rouge\">clang++<\/code>, since it is the system compiler. On <code class=\"language-plaintext highlighter-rouge\">Linux<\/code>, either <code class=\"language-plaintext highlighter-rouge\">clang++<\/code> or <code class=\"language-plaintext highlighter-rouge\">gcc<\/code> may be available depending on the distribution, but the workflow described in this post is written with <code class=\"language-plaintext highlighter-rouge\">clang++<\/code> as the preferred path. The language baseline is <code class=\"language-plaintext highlighter-rouge\">C++20<\/code>, and there is no implicit fallback to older standards. If a project requires <code class=\"language-plaintext highlighter-rouge\">C++20<\/code>, then that requirement should be stated explicitly in the build configuration rather than silently degraded by local machine defaults.<\/p>\n\n<p>Direct compiler invocation is not part of the normal workflow. It may still be useful for quick debugging or for isolating a small compilation issue, for example:<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>clang++ <span class=\"nt\">-std<\/span><span class=\"o\">=<\/span>c++20 <span class=\"nt\">-Wall<\/span> <span class=\"nt\">-Wextra<\/span> <span class=\"nt\">-O0<\/span> main.cpp\n<\/code><\/pre><\/div><\/div>\n\n<p>However, this kind of command should be treated as a temporary debugging tool rather than the standard way to build the project. More importantly, compiler flags should not be maintained manually across shell history, editor settings, and local notes. The ownership rule here is strict: compile flags belong to the build system, and in this setup that means <code class=\"language-plaintext highlighter-rouge\">CMake<\/code>.<\/p>\n\n<h2 id=\"build-system\">Build System<\/h2>\n\n<p>The build system is therefore the real center of the environment. This post uses <a href=\"https:\/\/cmake.org\/\"><code class=\"language-plaintext highlighter-rouge\">CMake<\/code><\/a> as the single source of truth, with <a href=\"https:\/\/ninja-build.org\/\"><code class=\"language-plaintext highlighter-rouge\">Ninja<\/code><\/a> as the preferred generator and <code class=\"language-plaintext highlighter-rouge\">Make<\/code> kept only as a legacy fallback. The standard workflow is:<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>cmake <span class=\"nt\">--preset<\/span> debug\ncmake <span class=\"nt\">--build<\/span> <span class=\"nt\">--preset<\/span> debug-build\n<\/code><\/pre><\/div><\/div>\n\n<p>An out-of-source build is required. Build artifacts, cache files, generated metadata, and configuration state should live in the build directory rather than being mixed into the source tree. This rule should hold regardless of project size. Even for a small executable, keeping source and build state separate makes the workflow easier to reason about, easier to clean, and easier to connect to tools that expect a stable build directory.<\/p>\n\n<p><code class=\"language-plaintext highlighter-rouge\">CMake<\/code> is not chosen here merely because it is popular. It is required because this setup needs a cross-platform abstraction layer, a reliable way to export compilation metadata for <code class=\"language-plaintext highlighter-rouge\">clangd<\/code>, and compatibility with the surrounding editor and tooling ecosystem. Once <code class=\"language-plaintext highlighter-rouge\">CMake<\/code> becomes the authority, the compiler stops being configured manually and starts executing instructions that come from one declarative source.<\/p>\n\n<p>For a modern CMake-based project, the recommended mode is <strong>preset-driven configuration<\/strong>. This model can be understood as two layers. The first is the configuration layer, which is file-driven and described by <a href=\"https:\/\/cmake.org\/cmake\/help\/latest\/manual\/cmake-presets.7.html\"><code class=\"language-plaintext highlighter-rouge\">CMakePresets.json<\/code><\/a>. That file declares the source directory, build directory, generator, and cache variables for each named configuration. The second is the execution layer, where the command line becomes the only operational entrypoint through the workflow shown above.<\/p>\n\n<p>This separation is important because it prevents configuration details from being scattered across shell history, editor tasks, and ad hoc scripts. Once a preset is defined, the CLI simply selects and executes it.<\/p>\n\n<p><code class=\"language-plaintext highlighter-rouge\">CMakePresets.json<\/code> should live at the project root and be treated as versioned project configuration rather than personal machine state. In practice, that means the project can define not only configure presets, but also build presets, test presets, and workflow presets. The configure preset captures how the build tree is created, the build preset captures how the generated build tree should be built, the test preset captures how <code class=\"language-plaintext highlighter-rouge\">ctest<\/code> should be run, and the workflow preset can chain multiple stages together. CMake exposes these layers directly through the CLI, for example <a href=\"https:\/\/cmake.org\/cmake\/help\/latest\/manual\/cmake.1.html\"><code class=\"language-plaintext highlighter-rouge\">cmake --build [&lt;dir&gt;] --preset &lt;preset&gt;<\/code><\/a> for builds, <a href=\"https:\/\/cmake.org\/cmake\/help\/latest\/manual\/ctest.1.html\"><code class=\"language-plaintext highlighter-rouge\">ctest --preset &lt;preset&gt;<\/code><\/a> for tests, and <a href=\"https:\/\/cmake.org\/cmake\/help\/latest\/manual\/cmake.1.html\"><code class=\"language-plaintext highlighter-rouge\">cmake --workflow --preset &lt;preset&gt;<\/code><\/a> for running a configured sequence of stages.<\/p>\n\n<p>This workflow support matters because it lets the project define not only static configuration, but also expected execution order. Instead of teaching every developer which exact configure, build, and test commands should be run in sequence, the project can encode that behavior once and expose it as a named workflow. In practice, that usually means a workflow such as <code class=\"language-plaintext highlighter-rouge\">debug-full<\/code> can configure, build, and test the same build tree consistently from one command.<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>cmake <span class=\"nt\">--workflow<\/span> <span class=\"nt\">--preset<\/span> debug-full\n<\/code><\/pre><\/div><\/div>\n\n<p>For test presets specifically, the project still needs to define tests in CMake itself, typically through <code class=\"language-plaintext highlighter-rouge\">enable_testing()<\/code> and <code class=\"language-plaintext highlighter-rouge\">add_test(...)<\/code>. Presets standardize how those tests are executed; they do not create test definitions automatically.<\/p>\n\n<p>The older pattern<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>cmake <span class=\"nt\">-S<\/span> <span class=\"nb\">.<\/span> <span class=\"nt\">-B<\/span> build <span class=\"nt\">-G<\/span> Ninja\n<\/code><\/pre><\/div><\/div>\n\n<p>is still valid CMake, but in this setup it should be treated as a legacy mode. It is acceptable for temporary experiments, CI scripts, or environments where presets are not available, but it should not be the primary developer workflow inside the project. If the project already knows its generator, build directory layout, and cache variables, those decisions should live in <code class=\"language-plaintext highlighter-rouge\">CMakePresets.json<\/code> rather than being retyped manually.<\/p>\n\n<p>A minimal preset file may look like this:<\/p>\n\n<div class=\"language-json highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"p\">{<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"version\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">6<\/span><span class=\"p\">,<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"cmakeMinimumRequired\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"major\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">3<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"minor\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">25<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"patch\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">0<\/span><span class=\"w\">\n  <\/span><span class=\"p\">},<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"configurePresets\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">[<\/span><span class=\"w\">\n    <\/span><span class=\"p\">{<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"displayName\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Debug\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"generator\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Ninja\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"binaryDir\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"${sourceDir}\/build\/debug\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"cacheVariables\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"CMAKE_BUILD_TYPE\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Debug\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"CMAKE_CXX_STANDARD\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"20\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"CMAKE_EXPORT_COMPILE_COMMANDS\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"ON\"<\/span><span class=\"w\">\n      <\/span><span class=\"p\">}<\/span><span class=\"w\">\n    <\/span><span class=\"p\">},<\/span><span class=\"w\">\n    <\/span><span class=\"p\">{<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"release\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"displayName\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Release\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"generator\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Ninja\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"binaryDir\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"${sourceDir}\/build\/release\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"cacheVariables\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"CMAKE_BUILD_TYPE\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Release\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"CMAKE_CXX_STANDARD\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"20\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"CMAKE_EXPORT_COMPILE_COMMANDS\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"ON\"<\/span><span class=\"w\">\n      <\/span><span class=\"p\">}<\/span><span class=\"w\">\n    <\/span><span class=\"p\">}<\/span><span class=\"w\">\n  <\/span><span class=\"p\">],<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"buildPresets\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">[<\/span><span class=\"w\">\n    <\/span><span class=\"p\">{<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug-build\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"configurePreset\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"jobs\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">8<\/span><span class=\"w\">\n    <\/span><span class=\"p\">},<\/span><span class=\"w\">\n    <\/span><span class=\"p\">{<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"release-build\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"configurePreset\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"release\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"jobs\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">8<\/span><span class=\"w\">\n    <\/span><span class=\"p\">}<\/span><span class=\"w\">\n  <\/span><span class=\"p\">],<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"testPresets\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">[<\/span><span class=\"w\">\n    <\/span><span class=\"p\">{<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug-test\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"configurePreset\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"output\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"outputOnFailure\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">true<\/span><span class=\"w\">\n      <\/span><span class=\"p\">},<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"execution\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"noTestsAction\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"error\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"stopOnFailure\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">true<\/span><span class=\"w\">\n      <\/span><span class=\"p\">}<\/span><span class=\"w\">\n    <\/span><span class=\"p\">},<\/span><span class=\"w\">\n    <\/span><span class=\"p\">{<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"release-test\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"configurePreset\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"release\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"output\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"outputOnFailure\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">true<\/span><span class=\"w\">\n      <\/span><span class=\"p\">}<\/span><span class=\"w\">\n    <\/span><span class=\"p\">}<\/span><span class=\"w\">\n  <\/span><span class=\"p\">],<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"workflowPresets\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">[<\/span><span class=\"w\">\n    <\/span><span class=\"p\">{<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug-full\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"steps\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">[<\/span><span class=\"w\">\n        <\/span><span class=\"p\">{<\/span><span class=\"w\">\n          <\/span><span class=\"nl\">\"type\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"configure\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n          <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug\"<\/span><span class=\"w\">\n        <\/span><span class=\"p\">},<\/span><span class=\"w\">\n        <\/span><span class=\"p\">{<\/span><span class=\"w\">\n          <\/span><span class=\"nl\">\"type\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"build\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n          <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug-build\"<\/span><span class=\"w\">\n        <\/span><span class=\"p\">},<\/span><span class=\"w\">\n        <\/span><span class=\"p\">{<\/span><span class=\"w\">\n          <\/span><span class=\"nl\">\"type\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"test\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n          <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug-test\"<\/span><span class=\"w\">\n        <\/span><span class=\"p\">}<\/span><span class=\"w\">\n      <\/span><span class=\"p\">]<\/span><span class=\"w\">\n    <\/span><span class=\"p\">}<\/span><span class=\"w\">\n  <\/span><span class=\"p\">]<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>The point of this file is not verbosity for its own sake. It turns configuration into versioned project state. Instead of every developer deciding locally where the build directory lives or whether compile commands should be exported, the project records those choices once and makes them reproducible.<\/p>\n\n<p>One useful way to think about presets is that they primarily handle <strong>selection and orchestration<\/strong>, not deep compilation policy. A preset chooses which build tree, generator, cache values, and workflow stages should be used. Lower-level compilation behavior, such as sanitizer instrumentation, is often cleaner when it is expressed through project options, target-level logic, or a dedicated toolchain file rather than by turning every preset into a bag of raw flags.<\/p>\n\n<h2 id=\"generators\">Generators<\/h2>\n\n<p>It is worth separating the concept of a <strong>build system<\/strong> from the concept of a <strong>generator<\/strong>, because they are related but not identical. <code class=\"language-plaintext highlighter-rouge\">CMake<\/code> is a meta-build system: it reads <code class=\"language-plaintext highlighter-rouge\">CMakeLists.txt<\/code> and preset configuration, resolves targets and cache variables, and then emits files for a concrete backend. That backend is called a generator. Officially, CMake supports command-line generators such as <code class=\"language-plaintext highlighter-rouge\">Ninja<\/code> and Unix Makefiles, as well as IDE-oriented generators such as Visual Studio and Xcode project files, all documented in the <a href=\"https:\/\/cmake.org\/cmake\/help\/latest\/manual\/cmake-generators.7.html\"><code class=\"language-plaintext highlighter-rouge\">cmake-generators(7)<\/code> manual<\/a>.<\/p>\n\n<p>The practical relationship is straightforward. The build system defines the project model, while the generator determines which concrete tool will execute that model. If the generator is <code class=\"language-plaintext highlighter-rouge\">Ninja<\/code>, CMake emits <code class=\"language-plaintext highlighter-rouge\">build.ninja<\/code> and related files. If the generator is Unix Makefiles, CMake emits <code class=\"language-plaintext highlighter-rouge\">Makefile<\/code>-based build scripts. The project semantics should remain the same, but the execution backend changes.<\/p>\n\n<p>In this article, <code class=\"language-plaintext highlighter-rouge\">Ninja<\/code> is preferred because it is fast, explicit, and fits well with a CLI-first workflow. <code class=\"language-plaintext highlighter-rouge\">Make<\/code> remains a fallback for older or more constrained environments, but it is not the primary recommendation. The key point is that developers should not think of the generator as a casual local preference. It is part of the configured build contract and therefore belongs in presets or other versioned build configuration.<\/p>\n\n<h2 id=\"compilation-metadata\">Compilation Metadata<\/h2>\n\n<p>The next part of the contract is compilation metadata. The key artifact is <code class=\"language-plaintext highlighter-rouge\">compile_commands.json<\/code>, which must be generated explicitly:<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>cmake <span class=\"nt\">--preset<\/span> debug\n<\/code><\/pre><\/div><\/div>\n\n<p>assuming that the preset enables <code class=\"language-plaintext highlighter-rouge\">CMAKE_EXPORT_COMPILE_COMMANDS=ON<\/code>. By default, <code class=\"language-plaintext highlighter-rouge\">compile_commands.json<\/code> is emitted into the corresponding build directory rather than the source root. In the preset example above, that means it would appear at <code class=\"language-plaintext highlighter-rouge\">build\/debug\/compile_commands.json<\/code> or <code class=\"language-plaintext highlighter-rouge\">build\/release\/compile_commands.json<\/code>. This matters because many editor integrations expect the file to exist either in the active build directory or to be symlinked from the project root. The important rule is that the file belongs to build state, not source state.<\/p>\n\n<p>This file is not a convenience feature for one editor plugin. It is the machine-readable description of how each translation unit is compiled. Each entry records the source file, include paths, compiler flags, macro definitions, and language standard associated with a concrete compile command. For <code class=\"language-plaintext highlighter-rouge\">clangd<\/code>, this database is foundational because <code class=\"language-plaintext highlighter-rouge\">clangd<\/code> operates by reading compilation metadata and constructing an AST from it. It does not reliably infer the real build configuration from the source tree alone.<\/p>\n\n<p>That is why missing or stale compilation metadata produces very specific failures: incorrect header resolution, broken go-to-definition, false diagnostics, or symbols that appear to vanish in the editor even though the project still builds. In many teams this is misdiagnosed as an editor problem, but in practice it usually means the compilation database is absent, outdated, or inconsistent with the actual build directory.<\/p>\n\n<h2 id=\"dependency-strategy\">Dependency Strategy<\/h2>\n\n<p>Dependency strategy is intentionally kept conservative in the baseline setup. No global dependency manager is required. System package managers such as <code class=\"language-plaintext highlighter-rouge\">brew<\/code> or <code class=\"language-plaintext highlighter-rouge\">apt<\/code> are acceptable for installing the toolchain itself. <code class=\"language-plaintext highlighter-rouge\">vcpkg<\/code> and <code class=\"language-plaintext highlighter-rouge\">Conan<\/code> are both valid options when a project needs stronger dependency management, but they are treated as optional extensions rather than part of the core environment. <code class=\"language-plaintext highlighter-rouge\">vcpkg<\/code> brings a relatively heavy integration footprint, while <code class=\"language-plaintext highlighter-rouge\">Conan<\/code> is better suited to full dependency graph management.<\/p>\n\n<p>The current decision is to exclude dependency management from the baseline configuration on purpose. The reason is not that these tools are bad, but that the minimal setup should reduce external state and keep reproducibility centered on the build system itself. Regardless of which package strategy a real project eventually adopts, dependencies should not alter the semantics of the build system, silently rewrite compiler invocation rules, or break LSP correctness.<\/p>\n\n<h2 id=\"build-type-model\">Build Type Model<\/h2>\n\n<p>Finally, the build type model must also be explicit. A <code class=\"language-plaintext highlighter-rouge\">Debug<\/code> configuration should keep optimization disabled, symbols enabled, and optionally allow sanitizers such as <code class=\"language-plaintext highlighter-rouge\">ASAN<\/code> or <code class=\"language-plaintext highlighter-rouge\">UBSAN<\/code>. A <code class=\"language-plaintext highlighter-rouge\">Release<\/code> configuration should enable optimization such as <code class=\"language-plaintext highlighter-rouge\">O2<\/code> or <code class=\"language-plaintext highlighter-rouge\">O3<\/code>, minimize or strip symbols when appropriate, and typically disable runtime assertions. These two modes should live in distinct build directories rather than being switched in place. In other words, <code class=\"language-plaintext highlighter-rouge\">Debug<\/code> and <code class=\"language-plaintext highlighter-rouge\">Release<\/code> are not just flags; they are separate build states with different semantics for compilation, execution, and debugging.<\/p>\n\n<p>A practical baseline looks like this:<\/p>\n\n<table>\n  <thead>\n    <tr>\n      <th>Build Type<\/th>\n      <th>Optimization<\/th>\n      <th>Debug Symbols<\/th>\n      <th>Sanitizers<\/th>\n      <th>Assertions<\/th>\n      <th>Build Directory<\/th>\n    <\/tr>\n  <\/thead>\n  <tbody>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">Debug<\/code><\/td>\n      <td><code class=\"language-plaintext highlighter-rouge\">O0<\/code><\/td>\n      <td>enabled<\/td>\n      <td>optional<\/td>\n      <td>enabled<\/td>\n      <td><code class=\"language-plaintext highlighter-rouge\">build\/debug<\/code><\/td>\n    <\/tr>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">Release<\/code><\/td>\n      <td><code class=\"language-plaintext highlighter-rouge\">O2<\/code> \/ <code class=\"language-plaintext highlighter-rouge\">O3<\/code><\/td>\n      <td>minimal<\/td>\n      <td>disabled<\/td>\n      <td>typically disabled<\/td>\n      <td><code class=\"language-plaintext highlighter-rouge\">build\/release<\/code><\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n\n<p>Taken together, this layer defines a strict division of responsibilities: the build system defines compilation truth, the compiler executes that truth, tooling consumes the exported metadata, and the IDE does not define any build semantics on its own.<\/p>\n\n<hr \/>\n\n<h1 id=\"project-structure-contract\">Project Structure Contract<\/h1>\n\n<p>For a C++ project built around <code class=\"language-plaintext highlighter-rouge\">CMake<\/code>, <code class=\"language-plaintext highlighter-rouge\">clangd<\/code>, and <code class=\"language-plaintext highlighter-rouge\">CTest<\/code>, directory layout is not just an aesthetic choice. It affects how build metadata is generated, how headers are interpreted, how tests are registered, and how debugging artifacts remain separated from source state. The principle is simple: <strong>filesystem layout should follow toolchain assumptions, not personal preference<\/strong>.<\/p>\n\n<p>This matters because the build system, language server, debugger, and test runner all touch the same project tree, but they do not interpret every directory in the same way. <code class=\"language-plaintext highlighter-rouge\">CMake<\/code> needs a stable source root and predictable generated directories. <code class=\"language-plaintext highlighter-rouge\">clangd<\/code> relies on build metadata that points back into source and header trees. The debugger expects binaries and symbols to come from a known build output. <code class=\"language-plaintext highlighter-rouge\">CTest<\/code> needs test registration to be explicit rather than inferred from ad hoc file placement. A layout that looks tidy to one developer but violates these assumptions usually creates friction later.<\/p>\n\n<h2 id=\"baseline-layout\">Baseline Layout<\/h2>\n\n<p>For this setup, a practical baseline looks like this:<\/p>\n\n<div class=\"language-text highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>project\/\n\u251c\u2500\u2500 CMakeLists.txt\n\u251c\u2500\u2500 CMakePresets.json\n\u251c\u2500\u2500 src\/\n\u251c\u2500\u2500 include\/\n\u251c\u2500\u2500 tests\/\n\u251c\u2500\u2500 cmake\/\n\u2514\u2500\u2500 build\/\n<\/code><\/pre><\/div><\/div>\n\n<p>This structure is intentionally modest. It is large enough to separate public headers, implementation files, helper CMake modules, tests, and generated artifacts, but small enough that the project does not become over-architected before it has real complexity.<\/p>\n\n<h2 id=\"directory-semantics\">Directory Semantics<\/h2>\n\n<p>Before going further, it is useful to define what each directory is allowed to mean.<\/p>\n\n<p>The <code class=\"language-plaintext highlighter-rouge\">src\/<\/code> directory holds implementation files, typically <code class=\"language-plaintext highlighter-rouge\">.cpp<\/code> translation units and internal implementation headers when needed. It is not the place where the project\u2019s external contract is defined. In other words, <code class=\"language-plaintext highlighter-rouge\">src\/<\/code> describes how the software is implemented, not what it promises to other targets.<\/p>\n\n<p>The <code class=\"language-plaintext highlighter-rouge\">include\/<\/code> directory holds public interface headers. These headers define the external contract of the project and should not depend on private build tree artifacts or fragile internal layout assumptions. If a header under <code class=\"language-plaintext highlighter-rouge\">include\/<\/code> only works because some local build detail leaks into it, then the boundary between public API and implementation has already become unclear. In practice, many projects go one step further and place public headers under a namespaced path such as <code class=\"language-plaintext highlighter-rouge\">include\/&lt;project-name&gt;\/...<\/code> to reduce header name collisions and make the installation boundary more explicit.<\/p>\n\n<p>The <code class=\"language-plaintext highlighter-rouge\">tests\/<\/code> directory holds test code. Test sources should live there rather than being mixed into production directories, and they should be added to the build graph explicitly through <code class=\"language-plaintext highlighter-rouge\">CMake<\/code>. Just as importantly, test layout should not distort the production build graph. Tests exist to validate production code, not to redefine how that code is organized.<\/p>\n\n<p>The <code class=\"language-plaintext highlighter-rouge\">build\/<\/code> directory is fully generated state. It should never be committed, and it should never contain authoritative source code. Build outputs, cache files, generated metadata such as <code class=\"language-plaintext highlighter-rouge\">compile_commands.json<\/code>, object files, binaries, and test artifacts belong there precisely because they are derived state rather than maintained source state. The same rule applies to generated headers: if the build system creates headers during configuration or build, those files belong to the build tree rather than being written back into <code class=\"language-plaintext highlighter-rouge\">include\/<\/code> or <code class=\"language-plaintext highlighter-rouge\">src\/<\/code>.<\/p>\n\n<p>The <code class=\"language-plaintext highlighter-rouge\">cmake\/<\/code> directory is optional but useful once the project grows beyond a single <code class=\"language-plaintext highlighter-rouge\">CMakeLists.txt<\/code>. It is the right place for helper modules, reusable configuration fragments, and toolchain abstractions that should not clutter the project root. What it should not become is a second source tree with unclear ownership.<\/p>\n\n<h2 id=\"test-directory-organization\">Test Directory Organization<\/h2>\n\n<p>One part of project structure deserves separate discussion: test layout. In practice, C++ projects do <strong>not<\/strong> have one universal standard for how the <code class=\"language-plaintext highlighter-rouge\">tests\/<\/code> directory should be organized. Several patterns are common, and each solves a different scaling problem.<\/p>\n\n<p>The first pattern is a flat structure:<\/p>\n\n<div class=\"language-text highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>tests\/\n\u251c\u2500\u2500 test_a.cpp\n\u2514\u2500\u2500 test_b.cpp\n<\/code><\/pre><\/div><\/div>\n\n<p>This is the simplest model. Each file may become its own test target, or several files may be grouped into a single test binary. It works well for small utilities, compact libraries, and CLI-oriented projects where the test suite is still narrow enough that extra hierarchy would add more noise than value.<\/p>\n\n<p>The second pattern is a category-based structure:<\/p>\n\n<div class=\"language-text highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>tests\/\n\u251c\u2500\u2500 unit\/\n\u2514\u2500\u2500 integration\/\n<\/code><\/pre><\/div><\/div>\n\n<p>This structure separates tests by intent rather than by source file location. <code class=\"language-plaintext highlighter-rouge\">unit\/<\/code> usually contains isolated logic tests with minimal external dependency, while <code class=\"language-plaintext highlighter-rouge\">integration\/<\/code> contains broader tests that cross module boundaries, perform I\/O, or rely on heavier runtime setup. This model aligns naturally with CI pipelines because unit and integration stages often have different runtime and failure expectations.<\/p>\n\n<p>The third pattern is a source-mirrored structure:<\/p>\n\n<div class=\"language-text highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>src\/\n\u251c\u2500\u2500 engine\/\n\u2514\u2500\u2500 net\/\n\ntests\/\n\u251c\u2500\u2500 engine\/\n\u2514\u2500\u2500 net\/\n<\/code><\/pre><\/div><\/div>\n\n<p>Here the test tree mirrors the production tree. This is common in large or long-lived systems because it creates clearer ownership boundaries and makes it easier to map tests back to the corresponding production modules. It is especially useful when teams are organized around module boundaries rather than around one shared code pool.<\/p>\n\n<p>These patterns can be summarized briefly:<\/p>\n\n<table>\n  <thead>\n    <tr>\n      <th>Pattern<\/th>\n      <th>Shape<\/th>\n      <th>Strength<\/th>\n      <th>Typical Usage<\/th>\n    <\/tr>\n  <\/thead>\n  <tbody>\n    <tr>\n      <td>Flat<\/td>\n      <td><code class=\"language-plaintext highlighter-rouge\">tests\/test_x.cpp<\/code><\/td>\n      <td>minimal overhead<\/td>\n      <td>small tools, small libraries<\/td>\n    <\/tr>\n    <tr>\n      <td>Category-based<\/td>\n      <td><code class=\"language-plaintext highlighter-rouge\">tests\/unit<\/code>, <code class=\"language-plaintext highlighter-rouge\">tests\/integration<\/code><\/td>\n      <td>explicit test intent and CI alignment<\/td>\n      <td>modular applications, actively maintained repos<\/td>\n    <\/tr>\n    <tr>\n      <td>Source-mirrored<\/td>\n      <td><code class=\"language-plaintext highlighter-rouge\">tests\/&lt;module&gt;<\/code> matching <code class=\"language-plaintext highlighter-rouge\">src\/&lt;module&gt;<\/code><\/td>\n      <td>ownership and traceability<\/td>\n      <td>large teams, long-lived systems<\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n\n<p>An important engineering clarification is that <strong>directory structure does not define test execution behavior<\/strong>. The filesystem can help humans organize test code, but it does not decide how tests are built, grouped, filtered, or scheduled.<\/p>\n\n<p>Execution semantics come from the build and test layers instead. <code class=\"language-plaintext highlighter-rouge\">CMake<\/code> defines test targets and registration through mechanisms such as <code class=\"language-plaintext highlighter-rouge\">add_test(...)<\/code>. <code class=\"language-plaintext highlighter-rouge\">CTest<\/code> classifies and filters tests through names, labels, and preset configuration. CI systems then decide which subsets of those registered tests run in which pipeline stage. This is why two projects can have almost identical <code class=\"language-plaintext highlighter-rouge\">tests\/<\/code> directories but completely different execution behavior.<\/p>\n\n<p>That distinction is worth making explicit because developers often overestimate what a folder name means. A directory named <code class=\"language-plaintext highlighter-rouge\">integration\/<\/code> may suggest slower or broader tests, but that meaning only becomes operational when the build system and test runner encode it. The folder alone is only an organizational signal.<\/p>\n\n<p>For this article, the recommended default is the <strong>category-based<\/strong> structure:<\/p>\n\n<div class=\"language-text highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>tests\/\n\u251c\u2500\u2500 unit\/\n\u2514\u2500\u2500 integration\/\n<\/code><\/pre><\/div><\/div>\n\n<p>This is not because it is the only correct layout, but because it gives a good balance between scalability and simplicity. It scales better than a flat directory once the test suite grows, it avoids coupling test structure too tightly to the internal <code class=\"language-plaintext highlighter-rouge\">src\/<\/code> layout, and it maps naturally to CI-level separation when unit and integration tests need different execution policies. At the same time, it stays simple enough for local iteration and does not force a large-system ownership model onto a small or medium project.<\/p>\n\n<h2 id=\"constraint-model\">Constraint Model<\/h2>\n\n<p>Regardless of which test layout a project chooses, some constraints should remain invariant.<\/p>\n\n<p>Tests should be independent of the production include layout in the sense that they must not depend on accidental header placement or private build-directory leakage. Tests must be registered through the build system rather than discovered through ad hoc filesystem conventions alone. And tests should not rely on directory shape itself as the source of execution semantics. Layout helps organization; <code class=\"language-plaintext highlighter-rouge\">CMake<\/code> and <code class=\"language-plaintext highlighter-rouge\">CTest<\/code> define behavior.<\/p>\n\n<p>Taken together, this gives a clean separation of concerns:<\/p>\n\n<ul>\n  <li>filesystem layout is the organizational layer<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">CMake<\/code> is the build graph definition layer<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">CTest<\/code> is the execution classification layer<\/li>\n<\/ul>\n\n<p>These layers are intentionally decoupled. That decoupling leads to the principle that matters most for day-to-day engineering:<\/p>\n\n<blockquote>\n  <p>Test directory structure is an organizational convention; execution semantics are defined by CMake and CTest, not the filesystem.<\/p>\n<\/blockquote>\n\n<hr \/>\n\n<h1 id=\"debug-stack\">Debug Stack<\/h1>\n\n<h2 id=\"cli-debugging\">CLI Debugging<\/h2>\n\n<p>In practice, the debugging stack in this setup has three layers: the command-line debugger, runtime instrumentation, and the editor or IDE frontend. They are related, but they are not interchangeable. The command-line debugger controls program execution, sanitizers instrument runtime behavior, and the IDE only orchestrates those lower layers through a user interface.<\/p>\n\n<p>At the CLI layer, the primary debugger depends on platform conventions. On <code class=\"language-plaintext highlighter-rouge\">macOS<\/code>, the practical default is <code class=\"language-plaintext highlighter-rouge\">lldb<\/code>. On <code class=\"language-plaintext highlighter-rouge\">Linux<\/code>, <code class=\"language-plaintext highlighter-rouge\">gdb<\/code> remains the most common baseline, although <code class=\"language-plaintext highlighter-rouge\">lldb<\/code> is also a viable option in many environments. The important point is not which debugger binary is more fashionable, but that the debugger operates directly on the compiled program and its symbols rather than on source files alone.<\/p>\n\n<p>Before going further, the most important requirement is that debugging needs a real debug build. At minimum, that means debug symbols must be enabled and aggressive optimization should be avoided. In a CMake-driven workflow, this typically means using a <code class=\"language-plaintext highlighter-rouge\">Debug<\/code> preset or setting:<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"nt\">-DCMAKE_BUILD_TYPE<\/span><span class=\"o\">=<\/span>Debug\n<\/code><\/pre><\/div><\/div>\n\n<p>At the raw compiler level, the equivalent intent is usually expressed with flags such as:<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"nt\">-g<\/span> <span class=\"nt\">-O0<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>Without symbols, backtraces lose fidelity and variable inspection becomes unreliable. With too much optimization, the debugger may still run, but source-level stepping, local variable visibility, and breakpoint behavior become much harder to interpret.<\/p>\n\n<p>A typical <code class=\"language-plaintext highlighter-rouge\">lldb<\/code> workflow looks like this:<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>lldb .\/build\/debug\/app\n<\/code><\/pre><\/div><\/div>\n\n<p>From there, only a small core command set is needed for day-to-day work:<\/p>\n\n<table>\n  <thead>\n    <tr>\n      <th>Command<\/th>\n      <th>Purpose<\/th>\n    <\/tr>\n  <\/thead>\n  <tbody>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">b main<\/code><\/td>\n      <td>set a breakpoint<\/td>\n    <\/tr>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">run<\/code><\/td>\n      <td>start execution<\/td>\n    <\/tr>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">bt<\/code><\/td>\n      <td>show a stack trace<\/td>\n    <\/tr>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">frame variable<\/code><\/td>\n      <td>inspect local variables<\/td>\n    <\/tr>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">next<\/code><\/td>\n      <td>step over<\/td>\n    <\/tr>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">step<\/code><\/td>\n      <td>step into<\/td>\n    <\/tr>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">continue<\/code><\/td>\n      <td>resume execution<\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n\n<p>The key constraint is that the CLI debugger operates on the <strong>compiled binary plus debug symbols<\/strong>. It does not depend on IDE state, editor plugins, or active CMake runtime context after the build has already happened. In that sense, the debugger is downstream of the build system, not embedded inside it. However, debugger correctness still depends heavily on build-system decisions: symbols must actually be generated, <code class=\"language-plaintext highlighter-rouge\">Debug<\/code> builds must avoid stripping them, and the build directory used for debugging must be the same one that produced the binary being inspected.<\/p>\n\n<h2 id=\"sanitizers\">Sanitizers<\/h2>\n\n<p>The second layer is runtime instrumentation. Sanitizers are not traditional debuggers; they are compiler-assisted runtime checks that modify the generated binary so that invalid behavior is detected at execution time. Their purpose is to catch classes of bugs that can be difficult to inspect manually in a debugger, especially memory misuse and undefined behavior.<\/p>\n\n<p>Two sanitizers are especially common in modern C++ workflows. <code class=\"language-plaintext highlighter-rouge\">AddressSanitizer<\/code> (<code class=\"language-plaintext highlighter-rouge\">ASAN<\/code>) is used to detect heap and stack memory errors such as use-after-free, out-of-bounds access, and similar corruption patterns. <code class=\"language-plaintext highlighter-rouge\">UndefinedBehaviorSanitizer<\/code> (<code class=\"language-plaintext highlighter-rouge\">UBSAN<\/code>) is used to detect invalid or undefined language semantics such as problematic integer overflow, invalid casts, or alignment violations. In practice, these two are often enabled together:<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"nt\">-fsanitize<\/span><span class=\"o\">=<\/span>address,undefined\n<\/code><\/pre><\/div><\/div>\n\n<p>There are several ways to express this in a CMake-based workflow. The most direct method is to inject the sanitizer flags through preset cache variables such as <code class=\"language-plaintext highlighter-rouge\">CMAKE_C_FLAGS<\/code>, <code class=\"language-plaintext highlighter-rouge\">CMAKE_CXX_FLAGS<\/code>, and <code class=\"language-plaintext highlighter-rouge\">CMAKE_EXE_LINKER_FLAGS<\/code> in a preset like <code class=\"language-plaintext highlighter-rouge\">debug-asan<\/code>. This works, and it is useful as an introductory setup because the preset becomes self-contained and no <code class=\"language-plaintext highlighter-rouge\">CMakeLists.txt<\/code> changes are required. The downside is that raw flags become scattered through cache state, are harder to compose cleanly, and can end up polluting broader build configuration.<\/p>\n\n<p>A more project-oriented approach is to expose an explicit option such as <code class=\"language-plaintext highlighter-rouge\">ENABLE_ASAN<\/code> in <code class=\"language-plaintext highlighter-rouge\">CMakeLists.txt<\/code>, and then let a preset turn that option on. That model is closer to common engineering practice because the project declares sanitizer support intentionally rather than smuggling it in through ad hoc cache strings. It also composes better with multiple build profiles.<\/p>\n\n<p>A minimal example looks like this:<\/p>\n\n<div class=\"language-cmake highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"nb\">option<\/span><span class=\"p\">(<\/span>ENABLE_ASAN <span class=\"s2\">\"Enable AddressSanitizer\"<\/span> OFF<span class=\"p\">)<\/span>\n\n<span class=\"nb\">if<\/span><span class=\"p\">(<\/span>ENABLE_ASAN<span class=\"p\">)<\/span>\n  <span class=\"nb\">add_compile_options<\/span><span class=\"p\">(<\/span>-fsanitize=address,undefined<span class=\"p\">)<\/span>\n  <span class=\"nb\">add_link_options<\/span><span class=\"p\">(<\/span>-fsanitize=address,undefined<span class=\"p\">)<\/span>\n<span class=\"nb\">endif<\/span><span class=\"p\">()<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>And the preset only needs to activate the project-level option:<\/p>\n\n<div class=\"language-json highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"p\">{<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug-asan\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"inherits\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"cacheVariables\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"ENABLE_ASAN\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"ON\"<\/span><span class=\"w\">\n  <\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Another structured approach is to isolate sanitizer behavior in a dedicated file such as <code class=\"language-plaintext highlighter-rouge\">cmake\/toolchains\/asan.cmake<\/code>, and let the preset select it. This can work well in a reusable template because the preset stays focused on configuration selection while the extra file holds the compilation policy. In that model, responsibilities are separated as follows:<\/p>\n\n<ul>\n  <li>the preset selects configuration<\/li>\n  <li>the toolchain defines compilation behavior<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">CMakeLists.txt<\/code> defines the build graph<\/li>\n<\/ul>\n\n<p>That can be expressed with a very small dedicated file such as <code class=\"language-plaintext highlighter-rouge\">cmake\/toolchains\/asan.cmake<\/code>:<\/p>\n\n<div class=\"language-cmake highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c1\"># Minimal example: real projects may also need to cover shared\/module linker flags<\/span>\n<span class=\"nb\">set<\/span><span class=\"p\">(<\/span>CMAKE_C_FLAGS <span class=\"s2\">\"<\/span><span class=\"si\">${<\/span><span class=\"nv\">CMAKE_C_FLAGS<\/span><span class=\"si\">}<\/span><span class=\"s2\"> -fsanitize=address,undefined\"<\/span><span class=\"p\">)<\/span>\n<span class=\"nb\">set<\/span><span class=\"p\">(<\/span>CMAKE_CXX_FLAGS <span class=\"s2\">\"<\/span><span class=\"si\">${<\/span><span class=\"nv\">CMAKE_CXX_FLAGS<\/span><span class=\"si\">}<\/span><span class=\"s2\"> -fsanitize=address,undefined\"<\/span><span class=\"p\">)<\/span>\n<span class=\"nb\">set<\/span><span class=\"p\">(<\/span>CMAKE_EXE_LINKER_FLAGS <span class=\"s2\">\"<\/span><span class=\"si\">${<\/span><span class=\"nv\">CMAKE_EXE_LINKER_FLAGS<\/span><span class=\"si\">}<\/span><span class=\"s2\"> -fsanitize=address,undefined\"<\/span><span class=\"p\">)<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>and a preset that selects it:<\/p>\n\n<div class=\"language-json highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"p\">{<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"debug-asan\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"generator\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Ninja\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"binaryDir\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"${sourceDir}\/build\/debug-asan\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"toolchainFile\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"cmake\/toolchains\/asan.cmake\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"cacheVariables\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"CMAKE_BUILD_TYPE\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Debug\"<\/span><span class=\"w\">\n  <\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>For a reusable project template or a blog post that is trying to teach a disciplined setup, this model can be a useful advanced option. It scales naturally when additional variants such as <code class=\"language-plaintext highlighter-rouge\">UBSAN<\/code>, <code class=\"language-plaintext highlighter-rouge\">TSAN<\/code>, or <code class=\"language-plaintext highlighter-rouge\">LSAN<\/code> need to be introduced later, and it keeps the preset readable instead of turning it into a raw flag container. However, for many ordinary projects, a project-level option plus target-based compile and link settings is still the more common default.<\/p>\n\n<p>Sanitizers work best when debug symbols are enabled and optimization is kept low, usually at <code class=\"language-plaintext highlighter-rouge\">O0<\/code> or <code class=\"language-plaintext highlighter-rouge\">O1<\/code>. They can still be used in other configurations, but highly optimized binaries are generally less pleasant to diagnose because the resulting runtime and stack traces become harder to interpret.<\/p>\n\n<p>Their execution model is also different from normal observation-based debugging. When a sanitizer detects a violation, the program typically aborts immediately and prints a diagnostic stack trace. This is not a recovery-oriented workflow. The binary is being instrumented so that illegal behavior becomes explicit as soon as it occurs.<\/p>\n\n<p>That leads to the main constraint of this layer: sanitizers are <strong>not observational tools<\/strong>. They change program behavior by injecting checks into the compiled binary. That is exactly why they are effective, but it is also why they should be understood as a distinct debugging layer rather than as a prettier form of logging or tracing.<\/p>\n\n<p>The most useful summary is therefore this: sanitizers should be treated primarily as a <strong>toolchain-level concern<\/strong>, not merely as a build preset concern. Presets select and orchestrate configurations; toolchains and project logic define how the binary is actually built.<\/p>\n\n<h2 id=\"ide-debugging\">IDE Debugging<\/h2>\n\n<p>The third layer is IDE integration. In this setup, IDE debugging should be understood as a frontend for the CLI debugger, not as an independent execution system. Whether the interface is <code class=\"language-plaintext highlighter-rouge\">nvim-dap<\/code>, a graphical IDE, or another DAP-compatible frontend, the underlying execution still flows through a real debugger such as <code class=\"language-plaintext highlighter-rouge\">lldb<\/code> or <code class=\"language-plaintext highlighter-rouge\">gdb<\/code>.<\/p>\n\n<p>For a Neovim-based workflow, the typical components are <code class=\"language-plaintext highlighter-rouge\">nvim-dap<\/code>, a debug adapter such as <code class=\"language-plaintext highlighter-rouge\">lldb-vscode<\/code>, and the Debug Adapter Protocol itself. The execution chain can be summarized as:<\/p>\n\n<div style=\"text-align: center;\" class=\"mermaid\">\nflowchart LR\n    A[IDE \/ Editor Frontend]\n    B[DAP]\n    C[lldb \/ gdb]\n    D[Binary]\n\n    A --&gt; B --&gt; C --&gt; D\n<\/div>\n\n<p>This separation is important because it keeps responsibility boundaries clear. The IDE does not compile the code, does not define build flags, and does not replace the debugger. It only launches and orchestrates the debugger with a structured configuration.<\/p>\n\n<p>In practice, the debugger frontend still needs a few concrete facts: the executable path, the working directory, and consistent source paths that match the debug symbols embedded in the binary. Editor-side project metadata such as <code class=\"language-plaintext highlighter-rouge\">compile_commands.json<\/code> can help the surrounding development experience feel coherent, but actual source-level debug mapping depends primarily on the binary, its symbols, and path consistency between build and source trees.<\/p>\n\n<hr \/>\n\n<h1 id=\"integrate-with-astronvim\">Integrate with AstroNvim<\/h1>\n\n<p>At this point, integrating the project with <code class=\"language-plaintext highlighter-rouge\">AstroNvim<\/code> should not require redefining the toolchain. If the project already exports <code class=\"language-plaintext highlighter-rouge\">compile_commands.json<\/code>, separates build directories cleanly, and exposes a predictable debug binary, then the editor only needs to consume that information. This is the reason I describe AstroNvim here as a frontend rather than as a build environment: it becomes useful only after the build system has already expressed the project correctly.<\/p>\n\n<h2 id=\"clangd-integration\">clangd Integration<\/h2>\n\n<p>For C++ editing, the most important integration path is still <code class=\"language-plaintext highlighter-rouge\">clangd<\/code>. AstroNvim does not need to understand <code class=\"language-plaintext highlighter-rouge\">CMake<\/code> targets directly if <code class=\"language-plaintext highlighter-rouge\">clangd<\/code> can already see the correct compilation database. In the <a href=\"https:\/\/github.com\/HuRuilizhen\/async_logger\"><code class=\"language-plaintext highlighter-rouge\">async_logger<\/code><\/a> example project, that contract is made explicit in two places.<\/p>\n\n<p>First, the project exports compile commands through <code class=\"language-plaintext highlighter-rouge\">CMakePresets.json<\/code>. Second, the project includes a local <code class=\"language-plaintext highlighter-rouge\">.clangd<\/code> file that points <code class=\"language-plaintext highlighter-rouge\">clangd<\/code> to the active build directory:<\/p>\n\n<div class=\"language-yaml highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"na\">CompileFlags<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">CompilationDatabase<\/span><span class=\"pi\">:<\/span> <span class=\"s\">build\/debug<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>This small file matters because it removes ambiguity about which build tree the editor should treat as authoritative. In a project with multiple build directories, <code class=\"language-plaintext highlighter-rouge\">clangd<\/code> should not be left to guess. The editor experience becomes much more stable when the project tells <code class=\"language-plaintext highlighter-rouge\">clangd<\/code> exactly where the relevant compilation database lives.<\/p>\n\n<p>Once that is in place, AstroNvim can delegate normal code intelligence work to <code class=\"language-plaintext highlighter-rouge\">clangd<\/code>: go-to-definition, completion, diagnostics, symbol navigation, and refactoring assistance. None of that requires the editor to define include paths or compile flags manually. The editor is only surfacing the model already produced by the build system.<\/p>\n\n<h2 id=\"debugger-frontend-integration\">Debugger Frontend Integration<\/h2>\n\n<p>The same thin-frontend principle applies to debugging. AstroNvim does not create a separate debugging model for the project. It simply provides a UI over the debugger stack described in the previous section. As long as the project already has a valid debug build and a predictable executable path, the editor can launch the debugger without redefining anything fundamental.<\/p>\n\n<p>In the <code class=\"language-plaintext highlighter-rouge\">async_logger<\/code> project, the build system already defines runtime output under <code class=\"language-plaintext highlighter-rouge\">${CMAKE_BINARY_DIR}\/bin<\/code>, and the presets define a deterministic build root such as <code class=\"language-plaintext highlighter-rouge\">build\/debug<\/code>. That means the editor-side debug configuration can stay simple: it only needs to know which binary to launch, which working directory to use, and whether stop-on-entry is desired. This is exactly the kind of integration that stays maintainable over time because the editor is consuming project state, not inventing it.<\/p>\n\n<h2 id=\"community-pack-and-lightweight-configuration\">Community Pack and Lightweight Configuration<\/h2>\n\n<p>One useful detail from the current setup is that the AstroNvim side is intentionally light. The active Neovim community imports include the C++ pack directly. In my current configuration, this lives in <a href=\"https:\/\/github.com\/HuRuilizhen\/astronvim-configuration\/blob\/main\/lua\/community.lua\"><code class=\"language-plaintext highlighter-rouge\">lua\/community.lua<\/code><\/a>, and the imported community modules come from the <a href=\"https:\/\/astronvim.github.io\/astrocommunity\/\">AstroCommunity catalog<\/a>:<\/p>\n\n<div class=\"language-lua highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c1\">-- lua\/community.lua<\/span>\n<span class=\"k\">return<\/span> <span class=\"p\">{<\/span>\n  <span class=\"s2\">\"AstroNvim\/astrocommunity\"<\/span><span class=\"p\">,<\/span>\n  <span class=\"p\">{<\/span> <span class=\"n\">import<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">\"astrocommunity.pack.lua\"<\/span> <span class=\"p\">},<\/span>\n  <span class=\"p\">{<\/span> <span class=\"n\">import<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">\"astrocommunity.pack.cpp\"<\/span> <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>That is enough to bring in a reasonable baseline for the language ecosystem without turning the editor configuration into the place where project semantics are defined. In other words, the community pack helps with editor capability, but it does not replace the role of <code class=\"language-plaintext highlighter-rouge\">CMake<\/code>, <code class=\"language-plaintext highlighter-rouge\">clangd<\/code>, or the debugger itself.<\/p>\n\n<p>This distinction is worth preserving. Tool installation and editor conveniences belong to the editor layer. Compilation truth, test registration, and debug correctness belong to the project layer. The more these responsibilities remain separated, the easier the setup is to reason about and reproduce.<\/p>\n\n<h2 id=\"what-astronvim-actually-needs\">What AstroNvim Actually Needs<\/h2>\n\n<p>Looking at the <code class=\"language-plaintext highlighter-rouge\">async_logger<\/code> project, the editor integration works well because the project already satisfies a short list of requirements:<\/p>\n\n<ul>\n  <li>a valid <code class=\"language-plaintext highlighter-rouge\">CMakePresets.json<\/code><\/li>\n  <li>an exported <code class=\"language-plaintext highlighter-rouge\">compile_commands.json<\/code><\/li>\n  <li>a <code class=\"language-plaintext highlighter-rouge\">.clangd<\/code> file that points to the intended compilation database<\/li>\n  <li>a deterministic build directory such as <code class=\"language-plaintext highlighter-rouge\">build\/debug<\/code><\/li>\n  <li>a predictable binary output path for debugging<\/li>\n<\/ul>\n\n<p>After these conditions are met, AstroNvim has very little left to invent. It can attach <code class=\"language-plaintext highlighter-rouge\">clangd<\/code>, surface diagnostics and navigation, and act as a DAP frontend for the debugger. That is the real integration story here: a good AstroNvim workflow for C++ is mostly a good CMake project with a thin editor frontend.<\/p>\n\n<hr \/>\n\n<p>Related Posts \/ Websites \ud83d\udc47<\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/github.com\/HuRuilizhen\/async_logger\">GitHub - HuRuilizhen\/async_logger<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/github.com\/HuRuilizhen\/astronvim-configuration\">GitHub - HuRuilizhen\/astronvim-configuration<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/astronvim.github.io\/astrocommunity\/\">AstroNvim - AstroCommunity<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/github.com\/AstroNvim\/astrocommunity\/tree\/main\/lua\/astrocommunity\/pack\/cpp\">GitHub - AstroCommunity C++ Pack<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/clangd.llvm.org\/config\">clangd - Configuration<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/cmake.org\/cmake\/help\/latest\/manual\/cmake-presets.7.html\">CMake - cmake-presets(7)<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/cmake.org\/cmake\/help\/latest\/manual\/cmake-generators.7.html\">CMake - cmake-generators(7)<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/cmake.org\/cmake\/help\/latest\/manual\/ctest.1.html\">CMake - ctest(1)<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/lldb.llvm.org\/\">LLVM - LLDB Documentation<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/github.com\/mfussenegger\/nvim-dap\">GitHub - mfussenegger\/nvim-dap<\/a><\/p>\n","pubDate":"Wed, 01 Jul 2026 00:00:00 +0000","link":"https:\/\/huruilizhen.github.io\/A-Productive-Cpp-Development-Environment","guid":"https:\/\/huruilizhen.github.io\/A-Productive-Cpp-Development-Environment","category":["cpp","development","package-release","best-practices"]},{"title":"Designing a Production-Ready Python Package: Structure, Tooling, and Automation","description":"<p>Python packaging has long been shaped by a fragmented ecosystem of loosely connected tools\u2014linters, formatters, build systems, and release workflows\u2014each solving a piece of the problem but rarely forming a coherent whole. Recent developments, particularly the rise of tools like Ruff and the standardization around <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code>, are beginning to change this landscape. This article presents a practical, end-to-end approach to designing a production-ready Python package, covering project structure, modern tooling choices, and CI\/CD automation, with an emphasis on clarity, consistency, and maintainability.<\/p>\n\n<!--more-->\n\n<h2 id=\"table-of-contents\">Table of Contents<\/h2>\n\n<!--toc:start-->\n<ul>\n  <li><a href=\"#table-of-contents\">Table of Contents<\/a><\/li>\n  <li><a href=\"#introduction\">Introduction<\/a>\n    <ul>\n      <li><a href=\"#the-problem\">The Problem<\/a><\/li>\n      <li><a href=\"#goals\">Goals<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#the-evolution-of-python-tooling\">The Evolution of Python Tooling<\/a>\n    <ul>\n      <li><a href=\"#fragmented-era\">Fragmented Era<\/a><\/li>\n      <li><a href=\"#early-attempts-at-standardization-black\">Early Attempts at Standardization: Black<\/a><\/li>\n      <li><a href=\"#toward-unification-ruff\">Toward Unification: Ruff<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#project-structure\">Project Structure<\/a>\n    <ul>\n      <li><a href=\"#src-vs-flat\">src vs flat<\/a><\/li>\n      <li><a href=\"#package-organization\">Package Organization<\/a><\/li>\n      <li><a href=\"#tests\">tests<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#build-system\">Build System<\/a>\n    <ul>\n      <li><a href=\"#setuppy-pyproject\">setup.py \u2192 pyproject<\/a><\/li>\n      <li><a href=\"#backends\">Backends<\/a><\/li>\n      <li><a href=\"#recommendation\">Recommendation<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#code-quality\">Code Quality<\/a>\n    <ul>\n      <li><a href=\"#traditional-issues\">Traditional Issues<\/a><\/li>\n      <li><a href=\"#ruff-replaces\">Ruff replaces<\/a><\/li>\n      <li><a href=\"#integration\">Integration<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#testing\">Testing<\/a>\n    <ul>\n      <li><a href=\"#what-to-test\">What to Test<\/a><\/li>\n      <li><a href=\"#testing-as-a-contract\">Testing as a Contract<\/a><\/li>\n      <li><a href=\"#interface-level-testing-cli-and-public-apis\">Interface-Level Testing: CLI and Public APIs<\/a><\/li>\n      <li><a href=\"#testing-in-ci\">Testing in CI<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#packaging-publishing\">Packaging &amp; Publishing<\/a>\n    <ul>\n      <li><a href=\"#build-artifacts\">Build Artifacts<\/a><\/li>\n      <li><a href=\"#publishing-to-pypi\">Publishing to PyPI<\/a><\/li>\n      <li><a href=\"#versioning-and-release-strategy\">Versioning and Release Strategy<\/a><\/li>\n      <li><a href=\"#common-pitfalls\">Common Pitfalls<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#cicd\">CI\/CD<\/a>\n    <ul>\n      <li><a href=\"#pipeline-design\">Pipeline Design<\/a><\/li>\n      <li><a href=\"#automated-releases\">Automated Releases<\/a><\/li>\n      <li><a href=\"#security-and-trust\">Security and Trust<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#conclusion\">Conclusion<\/a>\n<!--toc:end--><\/li>\n<\/ul>\n\n<hr \/>\n\n<h1 id=\"introduction\">Introduction<\/h1>\n\n<h2 id=\"the-problem\">The Problem<\/h2>\n\n<p>Python package development has historically relied on a collection of loosely connected tools, each addressing a specific concern\u2014linting, formatting, dependency management, building, and publishing. While this modularity provides flexibility, it often results in fragmented workflows, duplicated configuration, and inconsistent project conventions. Developers are left to manually assemble their own toolchains, making even simple tasks\u2014such as maintaining code quality or releasing a package\u2014more complex than necessary. The difficulty lies not in any single tool, but in the lack of a cohesive system that ties them together into a predictable and maintainable workflow.<\/p>\n\n<h2 id=\"goals\">Goals<\/h2>\n\n<p>This article aims to present a modern, production-oriented approach to Python packaging by establishing a coherent workflow that spans the entire lifecycle of a package\u2014from project structure and build configuration to code quality, testing, and automated releases. Rather than enumerating all available tools, the focus is on practical decisions and trade-offs, highlighting a streamlined toolchain centered around pyproject.toml and Ruff. The goal is to reduce complexity, improve consistency, and provide a clear, reproducible foundation for building and maintaining Python packages in real-world projects.<\/p>\n\n<hr \/>\n\n<h1 id=\"the-evolution-of-python-tooling\">The Evolution of Python Tooling<\/h1>\n\n<h2 id=\"fragmented-era\">Fragmented Era<\/h2>\n\n<p>Before the recent consolidation of Python tooling, package development typically involved assembling a collection of independent tools, each responsible for a narrow aspect of the workflow. Linting (e.g., <a href=\"https:\/\/flake8.pycqa.org\/en\/latest\/\">flake8<\/a>,  <a href=\"https:\/\/pylint.readthedocs.io\/en\/stable\/\">pylint<\/a>), formatting (e.g., <a href=\"https:\/\/black.readthedocs.io\/en\/stable\/\">Black<\/a>), import organization (e.g., <a href=\"https:\/\/here-be-pythons.readthedocs.io\/en\/latest\/python\/isort.html\">isort<\/a>), dependency management, and packaging (e.g., <a href=\"https:\/\/setuptools.pypa.io\/en\/latest\/\">setuptools<\/a>) were handled by separate utilities, often with overlapping responsibilities and inconsistent conventions.<\/p>\n<blockquote>\n  <p>Term LSP: The <strong>L<\/strong>anguage <strong>S<\/strong>erver <strong>P<\/strong>rotocol defines the protocol used between an editor or IDE and a language server that provides language features like auto complete, go to definition, find all references etc. For more infomation, visit: <a href=\"https:\/\/microsoft.github.io\/language-server-protocol\/\">https:\/\/microsoft.github.io\/language-server-protocol\/<\/a><\/p>\n<\/blockquote>\n\n<blockquote>\n  <p>Term Linter: A <strong>linter<\/strong> is a static code analysis tool used to provide diagnostics around programming errors, bugs, stylistic errors and suspicious constructs. Linters can be executed as a standalone program in a terminal, where it usually expects one or more input files to lint.<\/p>\n<\/blockquote>\n\n<p>This modular ecosystem provided flexibility, but at the cost of cohesion. Configuration was scattered across multiple files\u2014such as setup.py (packaging), setup.cfg (tool configuration), tox.ini (testing \/ environment), and requirements.txt (dependencies) \u2014making it difficult to maintain a clear and unified project setup. Tool interactions were not always predictable, and developers frequently had to resolve conflicts between formatting rules or manually coordinate execution order across tools.<\/p>\n\n<p>As a result, the complexity of Python packaging did not stem from any individual tool, but from the need to compose many tools into a working system. The lack of a cohesive workflow meant that even routine tasks\u2014such as enforcing code quality or preparing a release\u2014required non-trivial effort and careful orchestration.<\/p>\n\n<h2 id=\"early-attempts-at-standardization-black\">Early Attempts at Standardization: Black<\/h2>\n\n<p>The introduction of <a href=\"https:\/\/black.readthedocs.io\/en\/stable\/\">Black<\/a> marked an important step toward reducing this fragmentation by standardizing one aspect of the development process: code formatting. By adopting a strictly opinionated approach with minimal configuration, Black eliminated many of the debates and inconsistencies surrounding code style, allowing teams to converge on a single, deterministic format.<\/p>\n\n<p>This shift demonstrated that developers were willing to trade configurability for consistency, especially when it simplified collaboration and reduced cognitive overhead. However, Black addressed only a single dimension of the broader problem. Linting, import management, packaging, and workflow orchestration remained distributed across separate tools, each with its own configuration and execution model.<\/p>\n\n<p>In this sense, Black can be seen as an early attempt at standardization\u2014one that proved the value of unification, but also highlighted the limitations of solving fragmentation at only one layer of the toolchain.<\/p>\n\n<h2 id=\"toward-unification-ruff\">Toward Unification: Ruff<\/h2>\n\n<p>More recent developments, particularly the emergence of <a href=\"https:\/\/docs.astral.sh\/ruff\/\">Ruff<\/a>, signal a broader shift from tool composition toward tool consolidation. Rather than focusing on a single concern, Ruff integrates multiple responsibilities\u2014such as linting, import sorting, and code modernization\u2014into a single, high-performance tool built on a unified internal architecture.<\/p>\n\n<p>By analyzing source code through a single parsing pipeline and applying a wide range of rules within the same execution context, Ruff significantly reduces both runtime overhead and configuration complexity. It also enables automatic fixes for many classes of issues, further streamlining the development workflow.<\/p>\n\n<p>More importantly, Ruff changes the structure of the toolchain itself. Instead of coordinating multiple tools with overlapping responsibilities, developers can rely on a more centralized and consistent system. When combined with the standardization of project configuration through pyproject.toml, this shift enables a more cohesive and maintainable packaging workflow.<\/p>\n\n<p>This evolution\u2014from fragmented tools, to partial standardization, to broader unification\u2014reflects a gradual movement toward simplicity at the system level, rather than improvements in isolated components.<\/p>\n\n<hr \/>\n\n<h1 id=\"project-structure\">Project Structure<\/h1>\n\n<h2 id=\"src-vs-flat\">src vs flat<\/h2>\n\n<p>A fundamental decision in structuring a Python package is whether to use a flat layout or a src-based layout. While both can work, they differ significantly in how imports are resolved during development and testing.<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># flat layout package example<\/span>\ndemo\/\n  foo\/\n    __init__.py\n  run.py\n\n<span class=\"c\"># src-based layout package example<\/span>\ndemo\/\n  src\/\n    foo\/\n      __init__.py\n  run.py\n<\/code><\/pre><\/div><\/div>\n\n<p>In a flat layout, the package directory resides at the project root, making it directly discoverable by Python\u2019s import system. Since the current working directory is included in <code class=\"language-plaintext highlighter-rouge\">sys.path<\/code>, imports such as <code class=\"language-plaintext highlighter-rouge\">import mypkg<\/code> succeed by resolving modules from the source tree itself. This behavior is convenient, but it can mask issues: code that works locally may fail after installation, as the import no longer relies on the source directory but on the installed package.<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># import path example<\/span>\n\u276f python3.11\nPython 3.11.15 <span class=\"o\">(<\/span>main, Mar  3 2026, 00:52:57<span class=\"o\">)<\/span> <span class=\"o\">[<\/span>Clang 17.0.0 <span class=\"o\">(<\/span>clang-1700.6.3.2<span class=\"o\">)]<\/span> on darwin\nType <span class=\"s2\">\"help\"<\/span>, <span class=\"s2\">\"copyright\"<\/span>, <span class=\"s2\">\"credits\"<\/span> or <span class=\"s2\">\"license\"<\/span> <span class=\"k\">for <\/span>more information.\n<span class=\"o\">&gt;&gt;&gt;<\/span> import sys\n<span class=\"o\">&gt;&gt;&gt;<\/span> sys.path\n<span class=\"o\">[<\/span>\n  <span class=\"s1\">''<\/span>, <span class=\"c\"># current working directory, where you run python script<\/span>\n  <span class=\"s1\">'\/opt\/homebrew\/Cellar\/python@3.11\/3.11.15\/Frameworks\/Python.framework\/Versions\/3.11\/lib\/python311.zip'<\/span>,\n  <span class=\"s1\">'\/opt\/homebrew\/Cellar\/python@3.11\/3.11.15\/Frameworks\/Python.framework\/Versions\/3.11\/lib\/python3.11'<\/span>,\n  <span class=\"s1\">'\/opt\/homebrew\/Cellar\/python@3.11\/3.11.15\/Frameworks\/Python.framework\/Versions\/3.11\/lib\/python3.11\/lib-dynload'<\/span>,\n  <span class=\"s1\">'\/opt\/homebrew\/lib\/python3.11\/site-packages'<\/span>\n<span class=\"o\">]<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>The <code class=\"language-plaintext highlighter-rouge\">src\/<\/code> layout addresses this by placing the package under a dedicated source directory (e.g., <code class=\"language-plaintext highlighter-rouge\">src\/mypkg<\/code>). In this setup, the project root is no longer sufficient for resolving imports, and the package must be installed (or explicitly added to the import path) before it can be used. This enforces a stricter and more realistic workflow, ensuring that imports behave consistently between local development and deployed environments.<\/p>\n\n<p>Beyond consistency, the <code class=\"language-plaintext highlighter-rouge\">src\/<\/code> layout also helps prevent import shadowing, where modules in the project unintentionally override standard library or third-party packages due to their presence in the working directory. By separating the source tree from Python\u2019s default import path, this class of subtle and environment-dependent bugs is effectively eliminated.<\/p>\n\n<blockquote>\n  <p>Tips: Flat layouts test whether code can be imported from the source tree; <code class=\"language-plaintext highlighter-rouge\">src\/<\/code> layouts test whether it works as an installed package. For production-oriented packages, the <code class=\"language-plaintext highlighter-rouge\">src\/<\/code> layout provides stronger guarantees around correctness and reproducibility, and is therefore generally preferred over the flat layout.<\/p>\n<\/blockquote>\n\n<h2 id=\"package-organization\">Package Organization<\/h2>\n\n<p>Once the project layout is established, the next concern is how to organize code within the package itself. A well-structured package should reflect logical boundaries in the system, rather than incidental implementation details.<\/p>\n\n<p>At a minimum, modules should be grouped by functionality, with each subpackage representing a coherent responsibility. This helps maintain a clear separation of concerns and avoids the accumulation of large, monolithic modules that are difficult to reason about. Deep and overly nested hierarchies should also be avoided, as they tend to increase cognitive overhead without providing meaningful structure.<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># anti-pattern example: layer-based \/ technical role-based<\/span>\nmypkg\/\n  models\/\n  services\/\n  utils\/\n  helpers\/\n\n<span class=\"c\"># anti-pattern example: over-engineered \/ over-nested<\/span>\nmypkg\/\n  core\/\n    domain\/\n      entities\/\n      services\/\n    infrastructure\/\n      adapters\/\n      repositories\/\n<\/code><\/pre><\/div><\/div>\n\n<p>A common anti-pattern is organizing code by technical roles rather than functional boundaries. For example, separating modules into <code class=\"language-plaintext highlighter-rouge\">models\/<\/code>, <code class=\"language-plaintext highlighter-rouge\">services\/<\/code>, and <code class=\"language-plaintext highlighter-rouge\">utils\/<\/code>. While this may appear structured, it often leads to fragmented logic, where a single feature is spread across multiple directories, increasing coupling and making changes harder to implement. Another frequent issue is the overuse of generic <code class=\"language-plaintext highlighter-rouge\">utils<\/code> or <code class=\"language-plaintext highlighter-rouge\">helpers<\/code> modules, which tend to accumulate unrelated functionality and become implicit dependency hubs. In contrast, organizing code by functionality keeps related components together, making the system easier to understand, maintain, and evolve.<\/p>\n\n<blockquote>\n  <p>Tips: Functional organization keeps related logic together, reducing the need to navigate across multiple layers to understand or modify a feature.<\/p>\n<\/blockquote>\n\n<p>Equally important is defining a stable public interface. The top-level package (typically via <code class=\"language-plaintext highlighter-rouge\">__init__.py<\/code>) should expose only the components intended for external use, while internal modules remain encapsulated. This distinction between public and private APIs becomes especially valuable as the package evolves, allowing internal refactoring without breaking downstream users.<\/p>\n\n<p>For projects that include command-line interfaces, it is often useful to isolate CLI-related code into a dedicated module (e.g., <code class=\"language-plaintext highlighter-rouge\">cli\/<\/code> or <code class=\"language-plaintext highlighter-rouge\">__main__.py<\/code>), keeping it separate from core application logic. This separation ensures that the core package remains reusable as a library, independent of how it is invoked.<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># functionality-based package structure example <\/span>\nsrc\/\n  mypkg\/\n    __init__.py\n\n    user\/\n      __init__.py\n      models.py\n      service.py\n\n    auth\/\n      __init__.py\n      service.py\n      tokens.py\n\n    cli\/\n      __init__.py\n      main.py\n\n    _internal\/\n      config.py\n      logging.py\n<\/code><\/pre><\/div><\/div>\n\n<p>In the above structure, related components, such as models, services, and helpers\u2014are colocated within the same feature directory, rather than being split across technical layers. This makes the codebase easier to navigate and reduces the need to traverse multiple modules to understand or modify a feature. Separating CLI logic into its own module further ensures that the core package remains reusable as a library, while internal utilities can be grouped under a clearly marked private namespace.<\/p>\n\n<p>In practice, an effective package structure is one that makes the codebase easy to navigate, minimizes coupling between components, and provides a clear entry point for both users and maintainers.<\/p>\n\n<h2 id=\"tests\">tests<\/h2>\n\n<p>Testing is an integral part of a production-ready package, and its structure should reinforce clarity, isolation, and ease of execution. In practice, pytest has become the de facto standard for Python testing due to its simplicity and flexibility, and is a natural choice for most projects.<\/p>\n\n<p>A common and effective approach is to place all tests in a dedicated <code class=\"language-plaintext highlighter-rouge\">tests\/<\/code> directory at the project root, rather than embedding them within the package itself. This separation keeps production code and verification logic distinct, avoids unintentionally packaging test code during distribution, and makes the project layout easier to reason about. It also aligns well with pytest\u2019s default discovery mechanisms, requiring minimal configuration.<\/p>\n\n<p>Within the <code class=\"language-plaintext highlighter-rouge\">tests\/<\/code> directory, test modules are typically organized to mirror the package structure, though this does not need to be strictly enforced. A loose correspondence is often sufficient to maintain navigability without introducing unnecessary rigidity. The goal is to make it straightforward to locate the tests associated with a given feature, while preserving flexibility as the codebase evolves.<\/p>\n\n<p>For projects that expose a command-line interface, it is important to treat the CLI as an external interface and test it accordingly. Rather than relying solely on internal function calls, tests should validate behavior through invocation mechanisms such as subprocess execution or CLI runners. This ensures that the interface behaves correctly from a user\u2019s perspective, independent of its internal implementation.<\/p>\n\n<p>Overall, a well-structured testing setup emphasizes isolation, discoverability, and alignment with real usage patterns, helping to ensure that the package behaves consistently across development and deployment environments.<\/p>\n\n<hr \/>\n\n<h1 id=\"build-system\">Build System<\/h1>\n\n<h2 id=\"setuppy--pyproject\">setup.py \u2192 pyproject<\/h2>\n\n<p>Historically, Python packaging relied on <code class=\"language-plaintext highlighter-rouge\">setup.py<\/code> as both a configuration file and an executable script. While flexible, this approach blurred the boundary between definition and execution, allowing arbitrary code to run during the build process. As a result, builds were often difficult to reason about, inconsistent across environments, and tightly coupled to specific tooling conventions.<\/p>\n\n<p>The introduction of <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code> marked a shift toward a more declarative and standardized packaging model. Instead of embedding logic in Python code, project metadata and build configuration are expressed in a structured format that can be interpreted uniformly by different tools. This separation enables a clearer contract between the project and the build system, reducing implicit behavior and improving reproducibility.<\/p>\n\n<p>Equally important, <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code> serves as a unified configuration entry point for the broader Python tooling ecosystem. Beyond packaging, tools for linting, formatting, and dependency management increasingly rely on the same file, reducing configuration sprawl and promoting consistency across the project.<\/p>\n\n<p>With setup.py, defining a package requires executing Python code. With pyproject.toml, the package is described as data, allowing build tools to operate without arbitrary code execution. In effect, this transition reframes packaging from an execution-driven process to a configuration-driven one. By standardizing how projects declare their structure and requirements, <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code> provides a more predictable and maintainable foundation for modern Python development.<\/p>\n\n<blockquote>\n  <p>Tips: <code class=\"language-plaintext highlighter-rouge\">setup.py<\/code> executes code to define a package; <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code> describes a package without executing code.<\/p>\n<\/blockquote>\n\n<h2 id=\"backends\">Backends<\/h2>\n\n<p>While <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code> defines the structure and metadata of a project, it does not perform the build itself. This responsibility is delegated to a build backend, which implements the logic required to transform source code into distributable artifacts such as wheels or source distributions. Common backends include <a href=\"https:\/\/setuptools.pypa.io\/en\/latest\/\">setuptools<\/a>, <a href=\"https:\/\/github.com\/pypa\/hatch\">hatchling<\/a>, and <a href=\"https:\/\/github.com\/python-poetry\/poetry-core\">poetry-core<\/a>.<\/p>\n\n<p>A build backend can be understood as the execution layer of the packaging system. It interprets the project definition provided in <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code> and carries out the necessary steps to build the package. Different backends may offer varying levels of abstraction and additional features, but they all conform to a common interface, allowing tools like pip to interact with them in a standardized way.<\/p>\n\n<p>This separation between definition and execution is a key aspect of modern Python packaging. By decoupling project configuration from build logic, the ecosystem allows developers to choose a backend that fits their needs without changing how the project is described. In other words, <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code> specifies what the project is, while the build backend determines how it is built.<\/p>\n\n<p>In practice, this model enables a more modular and extensible tooling landscape, where improvements in build systems can be adopted independently of project configuration, and where projects remain interoperable across different tools and environments.<\/p>\n\n<h2 id=\"recommendation\">Recommendation<\/h2>\n\n<p>Choosing a build system is less about individual tools and more about understanding the trade-offs between different design approaches. In practice, a good build setup should prioritize simplicity, predictability, and compatibility with the broader Python ecosystem. These criteria help ensure that the project remains easy to maintain, behaves consistently across environments, and integrates smoothly with standard tooling.<\/p>\n\n<p>From this perspective, build systems can be broadly divided into two categories. Minimal backends, such as setuptools or hatchling, focus on implementing the standard packaging interface with minimal abstraction. They align closely with the <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code> specification, offer greater transparency, and reduce the risk of tool-specific lock-in. This makes them well-suited for libraries and projects that prioritize stability and long-term maintainability.<\/p>\n\n<p>In contrast, integrated toolchains, such as poetry, provide a more opinionated and feature-rich experience by combining dependency management, packaging, and environment handling into a single workflow. While this can improve developer ergonomics, it also introduces additional abstraction and tighter coupling to the tool itself, which may increase complexity over time.<\/p>\n\n<p>In most cases, a minimal, standard-aligned backend is the preferable choice. It keeps the configuration surface small, avoids unnecessary indirection, and adheres closely to the evolving Python packaging standards.<\/p>\n\n<blockquote>\n  <p>Tips: Integrated toolchains remain a reasonable option when their additional features are explicitly needed, but they should be adopted with an understanding of the trade-offs involved.<\/p>\n<\/blockquote>\n\n<hr \/>\n\n<h1 id=\"code-quality\">Code Quality<\/h1>\n\n<h2 id=\"traditional-issues\">Traditional Issues<\/h2>\n\n<p>As discussed earlier, Python tooling has historically been fragmented across multiple specialized utilities. This fragmentation is particularly evident in the domain of code quality, where linting, formatting, and import organization are treated as separate concerns, each managed by an independent tool.<\/p>\n\n<p>While this separation provides flexibility, it also introduces coordination overhead. Developers must configure and run multiple tools in tandem, often dealing with overlapping responsibilities and subtle inconsistencies between them. As a result, maintaining code quality becomes less about enforcing standards and more about managing the interactions between tools.<\/p>\n\n<p>This context sets the stage for a shift toward consolidation, where code quality is treated as a unified concern rather than a collection of loosely connected processes.<\/p>\n\n<h2 id=\"ruff-replaces\">Ruff replaces<\/h2>\n\n<p>Rather than merely replacing individual tools, Ruff changes how code quality is enforced in practice. Traditional workflows relied on orchestrating multiple utilities\u2014each responsible for a specific task\u2014requiring developers to coordinate execution order, resolve conflicts, and maintain separate configurations.<\/p>\n\n<p>With Ruff, these concerns are consolidated into a single execution model. Linting, formatting, and import organization are applied through a unified interface, allowing code quality checks to be performed in a single pass. This reduces both the operational overhead and the potential for inconsistency between tools, shifting the focus from tool coordination to outcome consistency.<\/p>\n\n<p>This shift also affects how developers interact with the toolchain. Instead of running multiple commands or integrating several tools into CI pipelines, code quality enforcement becomes a streamlined step, often reducible to a single command. As a result, the workflow becomes more predictable and easier to automate, particularly in larger projects where consistency is critical.<\/p>\n\n<p>In this sense, Ruff represents not just a consolidation of functionality, but a simplification of the development workflow itself.<\/p>\n\n<h2 id=\"integration\">Integration<\/h2>\n\n<p>Integrating code quality tools into the development workflow is as important as choosing the tools themselves. The goal is not merely to run checks, but to ensure that code quality is enforced consistently and automatically across all stages of development.<\/p>\n\n<p>In modern Python projects, configuration is typically centralized in <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code>, allowing tools like Ruff to operate with minimal setup and without scattered configuration files. This provides a single source of truth for code quality rules, reducing ambiguity and simplifying maintenance.<\/p>\n\n<p>More importantly, code quality checks should be integrated into automated workflows rather than relying on manual execution. While local tooling, such as editor integrations or pre-commit hooks, can provide immediate feedback, they are inherently optional and can be bypassed. <strong>C<\/strong>ontinuous <strong>I<\/strong>ntegration pipelines, on the other hand, serve as an enforcement layer, ensuring that all changes meet the defined standards before they are merged.<\/p>\n\n<blockquote>\n  <p>Term <strong>C<\/strong>ontinuous <strong>I<\/strong>ntegration, CI: Frequently merge code and automatically validate it (build + test), ensuring the codebase is always in an integratable state. It ends with the production of an artifact.<\/p>\n<\/blockquote>\n\n<blockquote>\n  <p>Term <strong>C<\/strong>ontinuous <strong>D<\/strong>eployment, CD: Automatically deliver or deploy validated code to runtime environments, ensuring it is always ready to be released. It begins by taking over that artifact.<\/p>\n<\/blockquote>\n\n<p>In practice, this means that code quality becomes a required step in the development lifecycle, not a best-effort guideline. By embedding checks into CI, projects can guarantee consistency across contributors and environments, eliminating discrepancies between local setups and production code.<\/p>\n\n<p>Ultimately, effective integration shifts code quality from a developer responsibility to a system property, where adherence to standards is enforced automatically rather than manually maintained.<\/p>\n\n<hr \/>\n\n<h1 id=\"testing\">Testing<\/h1>\n\n<blockquote>\n  <p>Tips: Tests are not just for catching bugs, they define what correct behavior means and ensure that it remains stable over time.<\/p>\n<\/blockquote>\n\n<h2 id=\"what-to-test\">What to Test<\/h2>\n\n<p>A common mistake in testing is to focus on implementation details rather than observable behavior. For a production-ready package, tests should primarily target the public API\u2014the functions, classes, and interfaces that users interact with directly. This ensures that tests remain stable even as internal implementations evolve.<\/p>\n\n<p>Beyond the public interface, core logic should be thoroughly exercised, especially in areas where correctness is critical or where failures would have significant impact. Edge cases also deserve explicit attention, as they often represent the boundaries where assumptions break down and unexpected behavior emerges.<\/p>\n\n<p>In contrast, testing internal helpers or transient implementation details tends to introduce unnecessary coupling between tests and code structure. Such tests are more likely to break during refactoring without providing meaningful signal. A well-designed test suite, therefore, focuses on behavior rather than structure, validating what the system does rather than how it is implemented.<\/p>\n\n<h2 id=\"testing-as-a-contract\">Testing as a Contract<\/h2>\n\n<p>At a deeper level, tests serve not only as a mechanism for detecting defects, but as a formalization of expected behavior. In this sense, a test suite functions as a contract: it defines what the package guarantees to its users and establishes the boundaries within which changes are considered safe.<\/p>\n\n<p>This perspective becomes particularly important as a project evolves. When implementation details change\u2014whether for optimization, refactoring, or new features\u2014the test suite provides a consistent reference point, ensuring that externally visible behavior remains intact. As long as the contract is preserved, internal changes can be made with confidence.<\/p>\n\n<p>Viewing tests as a contract also clarifies their role in long-term maintenance. Rather than being a one-time verification step, tests become a living specification that documents intended behavior and guards against regression. This shifts the purpose of testing from reactive bug detection to proactive stability assurance, reinforcing the reliability of the package over time.<\/p>\n\n<h2 id=\"interface-level-testing-cli-and-public-apis\">Interface-Level Testing: CLI and Public APIs<\/h2>\n\n<p>A practical way to apply the principles of behavior-driven testing is to focus on interface boundaries. Tests should exercise the system through the same entry points that users interact with, rather than relying on internal implementation details.<\/p>\n\n<p>For libraries, this typically means testing the public API directly\u2014invoking exposed functions and classes as they are intended to be used. This approach ensures that tests validate observable behavior and remain stable even when internal structures change.<\/p>\n\n<p>For command-line tools, the boundary is the CLI itself. Instead of calling internal functions, tests should invoke the command-line interface as an external process, verifying outputs, exit codes, and side effects. This more closely reflects real-world usage and helps uncover issues that would not surface through direct function calls alone.<\/p>\n\n<p>By consistently testing at interface boundaries, projects can reduce coupling between tests and implementation, improve the reliability of test results, and ensure that the system behaves correctly from the user\u2019s perspective. This aligns testing with actual usage patterns, making it both more robust and more meaningful.<\/p>\n\n<h2 id=\"testing-in-ci\">Testing in CI<\/h2>\n\n<p>Defining what to test and how to structure tests is only part of the equation. To be effective, testing must be consistently enforced across all changes, which is where CI plays a central role.<\/p>\n\n<p>Local testing provides fast feedback, but it ultimately depends on developer discipline and can be bypassed, whether intentionally or unintentionally. CI, in contrast, establishes an automated and consistent execution environment where tests are run on every change, ensuring that all contributions are validated against the same standards.<\/p>\n\n<p>By integrating tests into the CI pipeline\u2014typically as a required step for pull requests\u2014projects can enforce correctness as a non-negotiable condition for merging code. This transforms testing from a best-effort practice into a guaranteed property of the development process, preventing regressions and maintaining stability over time.<\/p>\n\n<p>In this model, testing is no longer an isolated activity but an integral part of the delivery pipeline. Combined with automated code quality checks, it forms a comprehensive validation layer that ensures both correctness and consistency before code reaches production.<\/p>\n\n<hr \/>\n\n<h1 id=\"packaging--publishing\">Packaging &amp; Publishing<\/h1>\n\n<h2 id=\"build-artifacts\">Build Artifacts<\/h2>\n\n<p>The output of a Python packaging process is not the source code itself, but a set of distributable artifacts. The two primary formats are source distributions (sdist) and built distributions (wheel), each serving a different role in the ecosystem.<\/p>\n\n<p>A source distribution contains the raw project files and requires a build step during installation. In contrast, a wheel is a pre-built artifact that can be installed directly without executing the build process, making it faster and more predictable. For this reason, wheels have become the preferred distribution format in most cases, as they reduce installation overhead and eliminate variability introduced by local build environments.<\/p>\n\n<p>An important but often overlooked aspect of packaging is that the contents of these artifacts are not implicitly derived from the repository. Instead, they are determined by the build configuration and inclusion rules defined by the project. This means that not all files present in the source tree are guaranteed to be included in the final distribution, and conversely, unintended files may be packaged if not explicitly excluded. In practice, these inclusion rules are defined through the project\u2019s build configuration (e.g., in <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code>, see <a href=\"https:\/\/packaging.python.org\/en\/latest\/guides\/writing-pyproject-toml\/\">https:\/\/packaging.python.org\/en\/latest\/guides\/writing-pyproject-toml\/<\/a>), which ultimately determines what is shipped to users.<\/p>\n\n<div class=\"language-toml highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># pyproject configuration example<\/span>\n<span class=\"nn\">[tool.hatch.build.targets.wheel]<\/span>\n<span class=\"py\">packages<\/span> <span class=\"p\">=<\/span> <span class=\"nn\">[\"src\/mypkg\"]<\/span>\n\n<span class=\"nn\">[tool.hatch.build.targets.sdist]<\/span>\n<span class=\"py\">only-include<\/span> <span class=\"p\">=<\/span> <span class=\"p\">[<\/span>\n  <span class=\"s\">\"\/src\/mypkg\"<\/span><span class=\"p\">,<\/span>\n  <span class=\"s\">\"\/README.md\"<\/span><span class=\"p\">,<\/span>\n  <span class=\"s\">\"\/LICENSE\"<\/span><span class=\"p\">,<\/span>\n  <span class=\"s\">\"\/pyproject.toml\"<\/span><span class=\"p\">,<\/span>\n<span class=\"p\">]<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>As a result, building a package is not merely a mechanical step, but a process of curating what is actually shipped to users. Ensuring that the correct modules, data files, and metadata are included\u2014and nothing more\u2014is essential for producing reliable and reproducible distributions. In this sense, distribution artifacts represent a controlled snapshot of the project, rather than a direct reflection of the repository state.<\/p>\n\n<h2 id=\"publishing-to-pypi\">Publishing to PyPI<\/h2>\n\n<p>Once a package has been built into distributable artifacts, the next step is to make those artifacts available to users. In the Python ecosystem, this is typically done by publishing them to a package index such as the <a href=\"https:\/\/pypi.org\/\">Python Package Index (PyPI)<\/a>, which serves as the central distribution layer for Python packages.<\/p>\n\n<p>It is important to distinguish between building and publishing. The build step produces artifacts\u2014such as wheels and source distributions while publishing is the act of uploading those artifacts to an index where they can be discovered and installed. Tools like pip do not interact with source repositories directly; instead, they retrieve pre-built distributions from package indexes, resolving dependencies and selecting appropriate artifacts for the target environment.<\/p>\n\n<p>In practice, publishing involves uploading pre-built artifacts to the index, typically authenticated via API tokens rather than user credentials. This separation of build and distribution reinforces a more reliable workflow: artifacts are produced in a controlled environment and then distributed without modification.<\/p>\n\n<p>From a system perspective, PyPI functions as a registry rather than a build service. It does not generate packages, but hosts and serves them. This distinction ensures that installation remains fast and predictable, as it relies on already prepared artifacts rather than executing arbitrary build steps at install time.<\/p>\n\n<h2 id=\"versioning-and-release-strategy\">Versioning and Release Strategy<\/h2>\n\n<p>Versioning is often treated as a simple numbering scheme, but in practice it defines the contract between a package and its users. A version is not merely a label, it communicates compatibility guarantees and sets expectations about how the package can evolve over time.<\/p>\n\n<p>In this context, versioning should be approached as part of a broader release strategy. Changes to a package\u2014whether bug fixes, new features, or breaking modifications\u2014must be reflected consistently in version increments. This ensures that users can reason about upgrades and manage dependencies without unexpected regressions.<\/p>\n\n<p>Semantic versioning provides a widely adopted convention for expressing these guarantees, but its effectiveness depends on discipline rather than the specification itself. A version number is only meaningful if it accurately reflects the nature of the changes it represents. Inconsistent or misleading versioning erodes trust and makes dependency management significantly more difficult.<\/p>\n\n<blockquote>\n  <p>Term SemVer: SemVer is a versioning scheme that encodes compatibility guarantees into version numbers using the <code class=\"language-plaintext highlighter-rouge\">MAJOR.MINOR.PATCH<\/code> format. <code class=\"language-plaintext highlighter-rouge\">MAJOR<\/code> Changes: incremented when there are <strong>breaking changes<\/strong> (incompatible changes); <code class=\"language-plaintext highlighter-rouge\">MINOR<\/code> Changes: incremented when adding functionality in a backward-compatible manner; <code class=\"language-plaintext highlighter-rouge\">PATCH<\/code> Changes: incremented for backward-compatible bug fixes.<\/p>\n<\/blockquote>\n\n<p>A well-defined release strategy also considers how and when versions are published. This may include the use of pre-release versions for unstable features, clear boundaries for breaking changes, and alignment between version tags and published artifacts. By treating versioning as a deliberate and consistent practice, projects can provide a stable and predictable experience for downstream users.<\/p>\n\n<h2 id=\"common-pitfalls\">Common Pitfalls<\/h2>\n\n<p>A recurring class of issues in Python packaging arises from the gap between local development and actual distribution. Code that works correctly in a local environment does not necessarily behave the same way once packaged and installed, leading to failures that only surface after release.<\/p>\n\n<p>One common example is import-related errors caused by differences between the source tree and the installed package. When code is executed directly from the project directory, modules may be resolved through the local file structure rather than through the installed distribution. This can mask missing files or incorrect package layouts, resulting in import failures for end users.<\/p>\n\n<p>Another frequent issue is incomplete distribution contents. Since build artifacts are defined by explicit inclusion rules, required files\u2014such as data assets or auxiliary modules\u2014may be omitted if not properly configured. These omissions often go unnoticed during local development, but lead to runtime errors once the package is installed from a distribution.<\/p>\n\n<p>Dependency-related problems are equally prevalent. A package may rely on libraries that are available in the developer\u2019s environment but are not declared in its metadata. While the code appears to function correctly locally, installation in a clean environment exposes missing dependencies, causing immediate failures.<\/p>\n\n<p>Version inconsistencies can also introduce subtle but impactful issues. If version identifiers are not managed consistently across source code, build artifacts, and release tags, users may unknowingly install incorrect or mismatched versions. This undermines the reliability of versioning as a communication mechanism and complicates debugging and support.<\/p>\n\n<p>Finally, non-reproducible builds present a more systemic challenge. When build outputs depend on environmental factors\u2014such as dynamic versioning, implicit dependencies, or machine-specific configurations\u2014the same source code may produce different artifacts across environments. This lack of determinism makes it difficult to verify and trust published packages.<\/p>\n\n<p>Taken together, these pitfalls highlight a common theme: correctness in packaging is not defined by local success, but by the reliability and consistency of the distributed artifacts. Ensuring that a package behaves as expected after installation requires deliberate attention to structure, configuration, and reproducibility throughout the build and release process.<\/p>\n\n<hr \/>\n\n<h1 id=\"cicd\">CI\/CD<\/h1>\n\n<h2 id=\"pipeline-design\">Pipeline Design<\/h2>\n\n<p>A well-designed CI\/CD pipeline is not merely a sequence of automated tasks, but a system for enforcing guarantees about the software being produced. Each stage in the pipeline serves a distinct purpose, collectively ensuring that the final artifact is correct, complete, and ready for distribution.<\/p>\n\n<p>In the context of Python packaging, a typical pipeline can be understood as a progression of validation and transformation steps: linting, testing, building, and publishing. Linting enforces code quality and consistency, preventing stylistic and structural issues from entering the codebase. Testing verifies functional correctness, ensuring that the behavior of the package remains stable across changes. Building transforms the validated source code into distributable artifacts, capturing a reproducible snapshot of the project. Finally, publishing makes these artifacts available to users through a package index.<\/p>\n\n<p>The ordering of these stages is deliberate. Validation steps precede artifact creation to ensure that only correct code is packaged, while publishing is deferred until all guarantees have been satisfied. This sequencing reduces the risk of distributing broken or inconsistent artifacts and reinforces a clear boundary between development and release.<\/p>\n\n<p>Rather than treating the pipeline as an operational detail, it is more accurate to view it as an executable specification of the project\u2019s quality standards. By encoding expectations\u2014such as passing tests, consistent formatting, and reproducible builds\u2014directly into the pipeline, teams ensure that these standards are applied uniformly and automatically.<\/p>\n\n<p>In this sense, a CI\/CD pipeline does not simply run tasks; it defines what it means for a package to be releasable. Any artifact that emerges from the pipeline has, by construction, satisfied the conditions required for distribution, making the release process both predictable and trustworthy.<\/p>\n\n<p>In practice, these concepts are typically implemented using CI\/CD systems such as <a href=\"https:\/\/docs.github.com\/en\/actions\">GitHub Actions<\/a> or <a href=\"https:\/\/docs.gitlab.com\/user\/project\/quick_actions\/\">GitLab CI<\/a>. For concrete examples, refer to the official documentation or minimal workflow templates. Recommend reading: <a href=\"https:\/\/docs.pypi.org\/trusted-publishers\/\">PyPI Trusted Publishers<\/a>.<\/p>\n\n<h2 id=\"automated-releases\">Automated Releases<\/h2>\n\n<p>In a well-structured workflow, releasing a package is not a manual step, but a deterministic outcome of versioning and validation. Once a version is defined and the corresponding code has passed all pipeline checks, the release process can be fully automated.<\/p>\n\n<p>A common pattern is to treat version tags as the trigger for release. When a new version is introduced\u2014typically through a version bump and an associated tag\u2014the pipeline interprets this as an intent to publish. The same system that validates the code then proceeds to build the corresponding artifacts and publish them to the package index. In this model, versioning, validation, and distribution are tightly coupled, eliminating the need for ad hoc release procedures.<\/p>\n\n<p>This approach has several advantages. It ensures that every published version has passed the full set of quality checks, as release is contingent on a successful pipeline execution. It also removes ambiguity from the release process: there is a single, consistent path from version definition to artifact publication, reducing the likelihood of human error or inconsistent states.<\/p>\n\n<p>From a design perspective, automated releases shift responsibility from individuals to the system. Rather than relying on developers to remember the correct sequence of steps, the pipeline encodes the release logic directly. As a result, publishing becomes a predictable and repeatable operation, aligned with the guarantees established earlier in the workflow.<\/p>\n\n<p>In this sense, a release is not something that is performed, but something that is derived. Once the conditions are met\u2014correct versioning, passing validation, and successful artifact generation\u2014the system completes the process by making the package available to users.<\/p>\n\n<h2 id=\"security-and-trust\">Security and Trust<\/h2>\n\n<p>The reliability of a software package is not determined solely by its functionality, but also by the trustworthiness of its distribution process. Users depend on the assumption that the package they install corresponds exactly to the source that was validated and released, without unintended modifications or interference.<\/p>\n\n<p>In this context, security is not an isolated concern, but an integral part of the distribution pipeline. The same system that enforces correctness must also ensure that artifacts are published in a controlled and verifiable manner. This includes authenticating publishing actions, restricting who or what can trigger releases, and maintaining a clear linkage between source code, build outputs, and published versions.<\/p>\n\n<p>Modern workflows increasingly rely on token-based authentication and automated publishing mechanisms to reduce the risks associated with manual processes. By delegating publishing responsibilities to the CI\/CD system, projects minimize exposure to credential leakage and eliminate inconsistencies introduced by local environments. In particular, approaches such as trusted publishing establish a direct relationship between the build pipeline and the package index, removing the need for long-lived credentials and strengthening the integrity of the release process.<\/p>\n\n<p>From a broader perspective, trust emerges from consistency and transparency. A reproducible build process, a deterministic release pipeline, and a clearly defined versioning strategy together provide users with confidence that what they install is both authentic and reliable. Security, in this sense, is not a separate layer added after the fact, but a property of a well-designed system.<\/p>\n\n<p>Ultimately, a trustworthy package is one whose entire lifecycle\u2014from source code to published artifact\u2014is governed by explicit, verifiable rules. By embedding these guarantees into the pipeline itself, projects move beyond ad hoc practices and establish a foundation for secure and dependable software distribution.<\/p>\n\n<hr \/>\n\n<h1 id=\"conclusion\">Conclusion<\/h1>\n\n<p>Designing a production-ready Python package is less about selecting individual tools and more about establishing a coherent system for building, validating, and distributing software. Each component\u2014project structure, build configuration, code quality tooling, and CI\/CD\u2014contributes to a larger workflow that defines how a package is produced and maintained.<\/p>\n\n<p>At its core, packaging is centered around artifacts rather than source code. What users install is not the repository itself, but a curated and reproducible distribution that must be explicitly defined and consistently generated.<\/p>\n\n<p>Versioning extends this system by acting as a contract between the package and its users. Meaningful version numbers communicate compatibility guarantees and enable predictable upgrades, but only when applied with discipline and consistency.<\/p>\n\n<p>CI\/CD completes the picture by transforming these practices into enforceable rules. A well-designed pipeline ensures that only validated code becomes distributable artifacts, and that releases are a natural consequence of versioning rather than a separate, manual process.<\/p>\n\n<p>Ultimately, the reliability of a Python package emerges from the alignment of these elements. When structure, tooling, versioning, and automation are treated as parts of a unified system, the result is not only a functional package, but a predictable and trustworthy distribution process.<\/p>\n\n<hr \/>\n\n<p>Related Posts \/ Websites \ud83d\udc47<\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/packaging.python.org\/en\/latest\/tutorials\/packaging-projects\/\">PyPA - Packaging Python Projects<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/packaging.python.org\/en\/latest\/guides\/writing-pyproject-toml\/\">PyPA - Writing your <code class=\"language-plaintext highlighter-rouge\">pyproject.toml<\/code><\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/docs.pytest.org\/en\/latest\/goodpractices.html\">pytest - Good Integration Practices<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/docs.astral.sh\/ruff\/\">Ruff Documentation<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/packaging.python.org\/en\/latest\/guides\/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows\/\">PyPA - Publishing package distribution releases using GitHub Actions CI\/CD workflows<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/docs.github.com\/actions\/deployment\/security-hardening-your-deployments\/configuring-openid-connect-in-pypi\">GitHub Docs - Configuring OpenID Connect in PyPI<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/semver.org\/\">SemVer Specification<\/a><\/p>\n","pubDate":"Mon, 30 Mar 2026 00:00:00 +0000","link":"https:\/\/huruilizhen.github.io\/Designing-a-Production-Ready-Python-Package","guid":"https:\/\/huruilizhen.github.io\/Designing-a-Production-Ready-Python-Package","category":["python","development","package-release","best-practices"]},{"title":"Windows Terminal Configuration","description":"<p>The most exciting part after getting a new machine is configuration! In this post, I will share my configuration of Windows Terminal with PowerShell, focusing on the appearance settings, package management tools, command-line tools, and editor settings. I will walk you through the process of setting up a beautiful and functional terminal experience on Windows, which will greatly enhance your productivity and coding experience.<\/p>\n\n<!--more-->\n\n<h2 id=\"table-of-contents\">Table of Contents<\/h2>\n\n<ul>\n  <li><a href=\"#installation-and-startup\">Installation and Startup<\/a>\n    <ul>\n      <li><a href=\"#intalling-windows-terminal-and-powershell-7x\">Intalling Windows Terminal and PowerShell 7.x<\/a><\/li>\n      <li><a href=\"#windows-terminal-startup\">Windows Terminal Startup<\/a><\/li>\n      <li><a href=\"#basic-powershell-instructions\">Basic PowerShell Instructions<\/a><\/li>\n      <li><a href=\"#basic-powershell-profile-setup\">Basic PowerShell Profile Setup<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#package-management-tools\">Package Management Tools<\/a>\n    <ul>\n      <li><a href=\"#winget\">winget<\/a><\/li>\n      <li><a href=\"#chocolatey\">chocolatey<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#appearance-settings\">Appearance Settings<\/a>\n    <ul>\n      <li><a href=\"#font\">font<\/a><\/li>\n      <li><a href=\"#posh\">posh<\/a><\/li>\n      <li><a href=\"#terminal-icons\">terminal icons<\/a><\/li>\n      <li><a href=\"#completionpredictor\">CompletionPredictor<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#command-line-tools\">Command-Line Tools<\/a>\n    <ul>\n      <li><a href=\"#tldr---community-driven-man\">tldr - Community-driven man<\/a><\/li>\n      <li><a href=\"#bat---better-cat\">bat - Better cat<\/a><\/li>\n      <li><a href=\"#winfetch---system-info\">winfetch - System Info<\/a><\/li>\n      <li><a href=\"#dust---quick-du\">dust - Quick du<\/a><\/li>\n      <li><a href=\"#bottom---better-top\">bottom - Better top<\/a><\/li>\n      <li><a href=\"#fzf---fuzzy-finder\">fzf - Fuzzy finder<\/a><\/li>\n      <li><a href=\"#zoxide---smart-cd\">zoxide - Smart cd<\/a><\/li>\n      <li><a href=\"#yazi---tui-file-manager\">yazi - TUI file manager<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#development-settings\">Development Settings<\/a>\n    <ul>\n      <li><a href=\"#lazygit---tui-for-git-commands\">lazygit - TUI for git commands<\/a><\/li>\n      <li><a href=\"#lazydocker---tui-for-docker-commands\">lazydocker - TUI for docker commands<\/a><\/li>\n      <li><a href=\"#astronvim---distr-of-neovim\">astronvim - Distr of Neovim<\/a><\/li>\n      <li><a href=\"#openssh-server-and-client\">OpenSSH Server and Client<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#conclusion\">Conclusion<\/a><\/li>\n<\/ul>\n\n<hr \/>\n\n<h1 id=\"installation-and-startup\">Installation and Startup<\/h1>\n\n<h2 id=\"intalling-windows-terminal-and-powershell-7x\">Intalling Windows Terminal and PowerShell 7.x<\/h2>\n\n<p>To get started, ensure you have Windows Terminal. If not, you can download it from the <a href=\"https:\/\/learn.microsoft.com\/en-us\/windows\/terminal\/install\">Microsoft Terminal Install Page<\/a>. The default version of PowerShell that comes with Windows is PowerShell 5.1, but I recommend installing PowerShell 7.x for a better experience. For example, PowerShell 7.x provides access to the <code class=\"language-plaintext highlighter-rouge\">Get-PSSubsystem<\/code> cmdlet, which is not available in PowerShell 5.1. The most useful function for me is <code class=\"language-plaintext highlighter-rouge\">PSReadLine<\/code>, which can give you completion suggestions as you type (based on your history).<\/p>\n\n<p>Please refer to the <a href=\"https:\/\/learn.microsoft.com\/en-us\/powershell\/scripting\/install\/installing-powershell-on-windows?view=powershell-7.5\">Installing PowerShell 7 on Windows<\/a> guide for installation instructions. Usually, I install PowerShell via <a href=\"https:\/\/learn.microsoft.com\/en-us\/windows\/package-manager\/winget\/\">winget<\/a> using the command:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">winget<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nt\">--id<\/span><span class=\"w\"> <\/span><span class=\"nx\">Microsoft.Powershell<\/span><span class=\"w\"> <\/span><span class=\"nt\">--source<\/span><span class=\"w\"> <\/span><span class=\"nx\">winget<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>One thing to note is that the PROFILE variable in PowerShell 7.x points to a different location than in PowerShell 5.1. You can check your profile path by running:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"bp\">$PROFILE<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Just for reference, you can check current PowerShell Modules and PSSusbsytems with the following commands. And here are my current outputs:<\/p>\n\n<p><a name=\"powershell-modules-and-pssubsystems\"><\/a><\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># PowerShell Modules<\/span><span class=\"w\">\n<\/span><span class=\"err\">&gt;<\/span><span class=\"w\"> <\/span><span class=\"n\">Get-Module<\/span><span class=\"w\">\n\n<\/span><span class=\"n\">ModuleType<\/span><span class=\"w\"> <\/span><span class=\"nx\">Version<\/span><span class=\"w\">    <\/span><span class=\"nx\">PreRelease<\/span><span class=\"w\"> <\/span><span class=\"nx\">Name<\/span><span class=\"w\">                                <\/span><span class=\"nx\">ExportedCommands<\/span><span class=\"w\">\n<\/span><span class=\"o\">----------<\/span><span class=\"w\"> <\/span><span class=\"o\">-------<\/span><span class=\"w\">    <\/span><span class=\"o\">----------<\/span><span class=\"w\"> <\/span><span class=\"o\">----<\/span><span class=\"w\">                                <\/span><span class=\"o\">----------------<\/span><span class=\"w\">\n<\/span><span class=\"n\">Manifest<\/span><span class=\"w\">   <\/span><span class=\"nx\">7.0.0.0<\/span><span class=\"w\">               <\/span><span class=\"nx\">Microsoft.PowerShell.Management<\/span><span class=\"w\">     <\/span><span class=\"p\">{<\/span><span class=\"n\">Add-Content<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Clear-Content<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Clear-Item<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Clear-ItemProperty<\/span><span class=\"err\">\u2026<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"n\">Manifest<\/span><span class=\"w\">   <\/span><span class=\"nx\">7.0.0.0<\/span><span class=\"w\">               <\/span><span class=\"nx\">Microsoft.PowerShell.Utility<\/span><span class=\"w\">        <\/span><span class=\"p\">{<\/span><span class=\"n\">Add-Member<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Add-Type<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Clear-Variable<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Compare-Object<\/span><span class=\"err\">\u2026<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"n\">Script<\/span><span class=\"w\">     <\/span><span class=\"nx\">0.0<\/span><span class=\"w\">                   <\/span><span class=\"nx\">oh-my-posh-core<\/span><span class=\"w\">                     <\/span><span class=\"p\">{<\/span><span class=\"n\">Enable-PoshLineError<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Enable-PoshTooltips<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Enable-PoshTransientPrompt<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Set-PoshContext<\/span><span class=\"err\">\u2026<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"n\">Script<\/span><span class=\"w\">     <\/span><span class=\"nx\">2.3.6<\/span><span class=\"w\">                 <\/span><span class=\"nx\">PSReadLine<\/span><span class=\"w\">                          <\/span><span class=\"p\">{<\/span><span class=\"n\">Get-PSReadLineKeyHandler<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Get-PSReadLineOption<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Remove-PSReadLineKeyHandler<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Set-PSReadLineKeyHandler<\/span><span class=\"err\">\u2026<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"n\">Script<\/span><span class=\"w\">     <\/span><span class=\"nx\">0.11.0<\/span><span class=\"w\">                <\/span><span class=\"nx\">Terminal-Icons<\/span><span class=\"w\">                      <\/span><span class=\"p\">{<\/span><span class=\"n\">Add-TerminalIconsColorTheme<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Add-TerminalIconsIconTheme<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Format-TerminalIcons<\/span><span class=\"p\">,<\/span><span class=\"w\"> <\/span><span class=\"nx\">Get-TerminalIconsColorTheme<\/span><span class=\"err\">\u2026<\/span><span class=\"p\">}<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># PSSusbsytems<\/span><span class=\"w\">\n<\/span><span class=\"err\">&gt;<\/span><span class=\"w\"> <\/span><span class=\"n\">Get-PSSubsystem<\/span><span class=\"w\">\n\n<\/span><span class=\"n\">Kind<\/span><span class=\"w\">              <\/span><span class=\"nx\">SubsystemType<\/span><span class=\"w\">      <\/span><span class=\"nx\">IsRegistered<\/span><span class=\"w\"> <\/span><span class=\"nx\">Implementations<\/span><span class=\"w\">\n<\/span><span class=\"o\">----<\/span><span class=\"w\">              <\/span><span class=\"o\">-------------<\/span><span class=\"w\">      <\/span><span class=\"o\">------------<\/span><span class=\"w\"> <\/span><span class=\"o\">---------------<\/span><span class=\"w\">\n<\/span><span class=\"n\">CommandPredictor<\/span><span class=\"w\">  <\/span><span class=\"nx\">ICommandPredictor<\/span><span class=\"w\">         <\/span><span class=\"nx\">False<\/span><span class=\"w\"> <\/span><span class=\"p\">{}<\/span><span class=\"w\">\n<\/span><span class=\"n\">CrossPlatformDsc<\/span><span class=\"w\">  <\/span><span class=\"nx\">ICrossPlatformDsc<\/span><span class=\"w\">         <\/span><span class=\"nx\">False<\/span><span class=\"w\"> <\/span><span class=\"p\">{}<\/span><span class=\"w\">\n<\/span><span class=\"n\">FeedbackProvider<\/span><span class=\"w\">  <\/span><span class=\"nx\">IFeedbackProvider<\/span><span class=\"w\">          <\/span><span class=\"nx\">True<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"n\">General<\/span><span class=\"w\"> <\/span><span class=\"nx\">Feedback<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>We will discuss some of these modules later in the post. Some modules may require additional installation steps, but they provide great functionality.<\/p>\n\n<h2 id=\"windows-terminal-startup\">Windows Terminal Startup<\/h2>\n\n<p>Personally, I like to set Windows Terminal with transparent background and Quake Mode (slide down from the top of the screen with a hotkey). Go to the Windows Terminal Settings, <code class=\"language-plaintext highlighter-rouge\">Settings &gt; General &gt; Appearance &gt; Transparency<\/code>. Enable Quake Mode by following the instructions on the <a href=\"https:\/\/learn.microsoft.com\/en-us\/windows\/terminal\/tips-and-tricks?WT.mc_id=DT-MVP-5004452#quake-mode\">Microsoft Quake Mode Page<\/a>. You can also set the background image, font, and other settings. Here is my current profile settings in json format:<\/p>\n\n<div class=\"language-json highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"p\">{<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"startup\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"defaultProfile\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"PowerShell\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"defaultTerminalApplication\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Windows Terminal\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"language\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"English (United States)\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"defaultImeInputMode\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Default\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"launchOnMachineStartup\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">true<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"firstWindowPreference\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"openTabWithDefaultProfile\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"newInstanceBehavior\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"createNewWindow\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"launchSize\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"columns\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">120<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"rows\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">30<\/span><span class=\"w\">\n    <\/span><span class=\"p\">},<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"launchParameters\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"default\"<\/span><span class=\"w\">\n  <\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>My appearance settings for PowerShell:<\/p>\n\n<div class=\"language-json highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"p\">{<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"profiles\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"list\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">[<\/span><span class=\"w\">\n      <\/span><span class=\"p\">{<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"name\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"PowerShell\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"colorScheme\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Campbell\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"font\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n          <\/span><span class=\"nl\">\"face\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"MesloLGM Nerd Font\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n          <\/span><span class=\"nl\">\"size\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">12<\/span><span class=\"p\">,<\/span><span class=\"w\">\n          <\/span><span class=\"nl\">\"weight\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"normal\"<\/span><span class=\"w\">\n        <\/span><span class=\"p\">},<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"lineHeight\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mf\">1.2<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"cellWidth\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mf\">0.6<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"useAcrylic\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">false<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"opacity\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">60<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"cursorShape\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"bar\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"cursorColor\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"#FFFFFF\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"padding\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"8, 8, 8, 8\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"scrollbarState\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"visible\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"intenseTextStyle\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"bright\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"backgroundImage\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">null<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"backgroundImageStretchMode\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"none\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"experimental.retroTerminalEffect\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">false<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"antialiasingMode\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"cleartype\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"adjustIndistinguishableTextLightness\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"never\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"useAtlasEngine\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">true<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"useAcrylicMaterial\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">false<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"backgroundImageOpacity\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mf\">1.0<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"backgroundImageAlignment\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"center\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"cursorHeight\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">100<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"fontFeatures\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">[],<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"builtinGlyphs\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">true<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"experimental.retroTerminalEffect\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">false<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"useAcrylic\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">false<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"acrylicOpacity\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mf\">0.6<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"experimental.pixelShaderPath\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">null<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"intenseTextStyle\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"bright\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"backgroundImageOpacity\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mf\">1.0<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"backgroundImageStretchMode\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"none\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"backgroundImageAlignment\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"center\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"useAtlasEngine\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">true<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"experimental.retroTerminalEffect\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">false<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"backgroundImageOpacity\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mf\">1.0<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"useAcrylic\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">false<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"backgroundOpacity\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">60<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"experimental.pixelShaderPath\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">null<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"intenseTextStyle\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"bright\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"backgroundImage\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">null<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"experimental.retroTerminalEffect\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">false<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"builtinGlyphs\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">true<\/span><span class=\"p\">,<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"experimental.useColorEmoji\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">true<\/span><span class=\"w\">\n      <\/span><span class=\"p\">}<\/span><span class=\"w\">\n    <\/span><span class=\"p\">]<\/span><span class=\"w\">\n  <\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Also, if you want to use Windows Terminal effectively, shortcut keys and hotkeys can be very useful.<\/p>\n\n<ul>\n  <li><code class=\"language-plaintext highlighter-rouge\">ctrl+shift+c<\/code> to close the current tab<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">ctrl+shift+t<\/code> to open a new tab<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">ctrl+shift+n<\/code> to open a new window<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">ctrl+shift+m<\/code> to open marker mode<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">ctrl+tab<\/code> to switch tabs forward<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">ctrl+shift+tab<\/code> to switch tabs backwards<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">f11<\/code> to toggle fullscreen mode<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">ctrl+f12<\/code> to toggle focus mode<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">ctrl+shift+p<\/code> to open the command palette<\/li>\n<\/ul>\n\n<p>You can find more in <code class=\"language-plaintext highlighter-rouge\">Settings &gt; Actions<\/code>.<\/p>\n\n<h2 id=\"basic-powershell-instructions\">Basic PowerShell Instructions<\/h2>\n\n<p>Commands in PowerShell can be very different from those in traditional Unix-like shells. Tutorials for beginners can be found in the <a href=\"https:\/\/learn.microsoft.com\/en-us\/powershell\/scripting\/learn\/ps101\/01-getting-started?view=powershell-7.5\">PowerShell Documentation<\/a>. There is a principal called \u201cVerb-Noun\u201d for cmdlet naming conventions. For example, <code class=\"language-plaintext highlighter-rouge\">Get-Help<\/code>, <code class=\"language-plaintext highlighter-rouge\">Set-Location<\/code>, <code class=\"language-plaintext highlighter-rouge\">Get-Process<\/code>, etc. You can use <code class=\"language-plaintext highlighter-rouge\">Get-Help &lt;cmdlet-name&gt;<\/code> to get help on any cmdlet. For example:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># Get help for the Get-Process cmdlet<\/span><span class=\"w\">\n<\/span><span class=\"n\">Get-Help<\/span><span class=\"w\"> <\/span><span class=\"nx\">Get-Process<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># Get commands related to a specific noun<\/span><span class=\"w\">\n<\/span><span class=\"n\">Get-Command<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Noun<\/span><span class=\"w\"> <\/span><span class=\"err\">&lt;<\/span><span class=\"nx\">noun<\/span><span class=\"err\">&gt;<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># Get processes running on the system (ps equivalent)<\/span><span class=\"w\">\n<\/span><span class=\"n\">Get-Process<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># Get current directory (pwd equivalent)<\/span><span class=\"w\">\n<\/span><span class=\"n\">Get-Location<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># Get Items in the current directory (ls equivalent)<\/span><span class=\"w\">\n<\/span><span class=\"n\">Get-ChildItem<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Not that hard, right? Once you get used to it, PowerShell can be very powerful and flexible. But there is still a huge difference between PowerShell and traditional shells, so be sure to check out the documentation for more details. Especially, parameters and piping. Unlike traditional shells (such as CMD and Bash) that pass <strong>text<\/strong>, PowerShell passes rich \u00b7.NET\u00b7 <strong>objects<\/strong> that contain <strong>properties<\/strong> and <strong>methods<\/strong>. Here are some examples for reference:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># Piping example: Get processes and sort by CPU usage<\/span><span class=\"w\">\n<\/span><span class=\"n\">Get-Process<\/span><span class=\"w\"> <\/span><span class=\"o\">|<\/span><span class=\"w\"> <\/span><span class=\"n\">Sort-Object<\/span><span class=\"w\"> <\/span><span class=\"nx\">CPU<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Descending<\/span><span class=\"w\"> <\/span><span class=\"o\">|<\/span><span class=\"w\"> <\/span><span class=\"n\">Select-Object<\/span><span class=\"w\"> <\/span><span class=\"nt\">-First<\/span><span class=\"w\"> <\/span><span class=\"nx\">5<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># Parameter example: Get processes with a name containing \"chrome\" and stop them<\/span><span class=\"w\">\n<\/span><span class=\"n\">Get-Process<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"o\">*<\/span><span class=\"nx\">chrome<\/span><span class=\"o\">*<\/span><span class=\"w\"> <\/span><span class=\"o\">|<\/span><span class=\"w\"> <\/span><span class=\"n\">Stop-Process<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<h2 id=\"basic-powershell-profile-setup\">Basic PowerShell Profile Setup<\/h2>\n\n<p>Powershell profile is a file that contains commands and settings that are executed when you open a new PowerShell session. Consider it like a startup script for your PowerShell (.bashrc for Unix-like shells). You can add some basic commands and settings to your profile to make your life easier. Later, I will share my PowerShell profile setup in this post function by function. To open your profile, type <code class=\"language-plaintext highlighter-rouge\">&lt;editor&gt; $PROFILE<\/code> in PowerShell. Again, note that <code class=\"language-plaintext highlighter-rouge\">$PROFILE<\/code> is different for different PowerShell versions. So, make sure to replace it with the correct path.<\/p>\n\n<hr \/>\n\n<h1 id=\"package-management-tools\">Package Management Tools<\/h1>\n\n<p>Like apt for Ubuntu, yum for CentOS, brew for macOS, etc. Windows Terminal comes with a package manager called <strong>winget<\/strong> and a package manager called <strong>chocolatey<\/strong>. They are both very useful and easy to use. But focus on different use cases. I both of them for different purposes.<\/p>\n\n<h2 id=\"winget\">winget<\/h2>\n\n<p>Officially produced by Microsoft, it is lightweight, modern, and tightly integrated with the system. It is more like \u201cApp Store CLI\u201d for Windows, focusing on installing and managing free open source or store applications. Features:<\/p>\n\n<ul>\n  <li><strong>Official Installation<\/strong>: In most cases, the official installer (e.g., .msi, .exe) is called and then installed using its silent parameters. The more modern MSIX is also supported.<\/li>\n  <li><strong>Secure Source<\/strong>: Microsoft will perform some scanning and verification, and the source is relatively trustworthy.<\/li>\n  <li><strong>Simple Usage<\/strong>: The command line syntax is more modern and concise.<\/li>\n<\/ul>\n\n<p>You can find more information on the <a href=\"https:\/\/github.com\/microsoft\/winget-cli\">Winget GitHub Page<\/a> or check the official website at <a href=\"https:\/\/learn.microsoft.com\/en-us\/windows\/package-manager\/\">Windows Package Manager<\/a>. It is available for Windows 11 by default most of the time. You can also install it via the Windows Store by searching for \u201cWinget\u201d. For some useful commands, you can refer to the <a href=\"https:\/\/learn.microsoft.com\/en-us\/windows\/package-manager\/winget\/install?source=recommendations\">Winget documentation<\/a>.<\/p>\n\n<h2 id=\"chocolatey\">chocolatey<\/h2>\n\n<p>Chocolatey is an \u201cautomated software deployment platform\u201d for Windows that manages software as code. Features:<\/p>\n\n<ul>\n  <li><strong>Community Driven<\/strong> : Community projects, commercial company support. \u201cTrust lies in the community.\u201d The packages in the public repository are provided by different maintainers, and users need to make their own judgments.<\/li>\n  <li><strong>Powerful Installation<\/strong>: It completely relies on automated scripts (PowerShell) to download and install silently, and can handle very complex installation processes (such as entering serial numbers, copying files, etc.).<\/li>\n  <li><strong>Classic Syntax<\/strong>: The command line syntax is classic and powerful, and some commands are more complex.<\/li>\n<\/ul>\n\n<p>Download and install Chocolatey via the <a href=\"https:\/\/github.com\/chocolatey\/chocolatey\">Chocolatey GitHub Page<\/a> and official website at <a href=\"https:\/\/chocolatey.org\/install\">Chocolatey Installation<\/a>.  Run the following command:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">Set-ExecutionPolicy<\/span><span class=\"w\"> <\/span><span class=\"nx\">Bypass<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Scope<\/span><span class=\"w\"> <\/span><span class=\"nx\">Process<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Force<\/span><span class=\"p\">;<\/span><span class=\"w\"> <\/span><span class=\"p\">[<\/span><span class=\"n\">System.Net.ServicePointManager<\/span><span class=\"p\">]::<\/span><span class=\"n\">SecurityProtocol<\/span><span class=\"w\"> <\/span><span class=\"o\">=<\/span><span class=\"w\"> <\/span><span class=\"p\">[<\/span><span class=\"n\">System.Net.ServicePointManager<\/span><span class=\"p\">]::<\/span><span class=\"n\">SecurityProtocol<\/span><span class=\"w\"> <\/span><span class=\"o\">-bor<\/span><span class=\"w\"> <\/span><span class=\"nx\">3072<\/span><span class=\"p\">;<\/span><span class=\"w\"> <\/span><span class=\"n\">iex<\/span><span class=\"w\"> <\/span><span class=\"p\">((<\/span><span class=\"n\">New-Object<\/span><span class=\"w\"> <\/span><span class=\"nx\">System.Net.WebClient<\/span><span class=\"p\">)<\/span><span class=\"o\">.<\/span><span class=\"nf\">DownloadString<\/span><span class=\"p\">(<\/span><span class=\"s1\">'https:\/\/community.chocolatey.org\/install.ps1'<\/span><span class=\"p\">))<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<hr \/>\n\n<h1 id=\"appearance-settings\">Appearance Settings<\/h1>\n\n<p>Oh My Posh is a PowerShell prompt theme that is very popular (like oh my zsh in Windows). It is a very nice theme, with many different themes and styles. In this section, I will share my configuration and settings for Oh My Posh~<\/p>\n\n<h2 id=\"font\">font<\/h2>\n\n<p>Oh My Posh was designed to use <strong>Nerd Fonts<\/strong>. Nerd Fonts are popular fonts that are patched to include icons. To see the icons displayed in Oh My Posh, install a Nerd Font, and configure your terminal to use it. Installation and configuration instructions can be found on the <a href=\"https:\/\/ohmyposh.dev\/docs\/installation\/fonts\">Oh My Posh Fonts<\/a>. Installation by command:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">Install-PSResource<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">NerdFonts<\/span><span class=\"w\">\n<\/span><span class=\"n\">Import-Module<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">NerdFonts<\/span><span class=\"w\">\n\n<\/span><span class=\"n\">Install-NerdFont<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">FiraCode<\/span><span class=\"w\">   <\/span><span class=\"c\"># Tab completion works on name<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Configuration in Windows Terminal. This can be easily done by modifying the Windows Terminal settings (default shortcut: <code class=\"language-plaintext highlighter-rouge\">CTRL + SHIFT + ,<\/code>). In your settings.json file, add the <code class=\"language-plaintext highlighter-rouge\">font.face<\/code> attribute under the defaults attribute in profiles:<\/p>\n\n<div class=\"language-json highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"profiles\"<\/span><span class=\"p\">:<\/span><span class=\"w\">\n    <\/span><span class=\"p\">{<\/span><span class=\"w\">\n        <\/span><span class=\"nl\">\"defaults\"<\/span><span class=\"p\">:<\/span><span class=\"w\">\n        <\/span><span class=\"p\">{<\/span><span class=\"w\">\n            <\/span><span class=\"nl\">\"font\"<\/span><span class=\"p\">:<\/span><span class=\"w\">\n            <\/span><span class=\"p\">{<\/span><span class=\"w\">\n                <\/span><span class=\"nl\">\"face\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"MesloLGM Nerd Font\"<\/span><span class=\"w\">\n            <\/span><span class=\"p\">}<\/span><span class=\"w\">\n        <\/span><span class=\"p\">}<\/span><span class=\"w\">\n    <\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Configuration in Visual Studio Code is also supported. If you are using the JSON based settings, you will need to update the terminal.integrated.fontFamily value. Example in case of MesloLGM Nerd Font Nerd Font:<\/p>\n\n<div class=\"language-json highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"nl\">\"terminal.integrated.fontFamily\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"MesloLGM Nerd Font\"<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<h2 id=\"posh\">posh<\/h2>\n\n<p>So called \u201cThe most customizable and fastest prompt engine for any shell\u201d. Check the <a href=\"https:\/\/github.com\/JanDeDobbeleer\/oh-my-posh\">Oh My Posh GitHub Page<\/a> and official website at <a href=\"https:\/\/ohmyposh.dev\/\">Oh My Posh<\/a>. We can install it via chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">oh-my-posh<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>or install it via winget:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">winget<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">JanDeDobbeleer.OhMyPosh<\/span><span class=\"w\"> <\/span><span class=\"nt\">--source<\/span><span class=\"w\"> <\/span><span class=\"nx\">winget<\/span><span class=\"w\"> <\/span><span class=\"nt\">--scope<\/span><span class=\"w\"> <\/span><span class=\"nx\">user<\/span><span class=\"w\"> <\/span><span class=\"nt\">--force<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>For the <code class=\"language-plaintext highlighter-rouge\">PATH<\/code> to be reloaded, a restart of your terminal is advised. If oh-my-posh is not recognized as a command, you can run the installer again, or add it manually to your PATH. For example:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"nv\">$<\/span><span class=\"nn\">env<\/span><span class=\"p\">:<\/span><span class=\"nv\">Path<\/span><span class=\"w\"> <\/span><span class=\"o\">+=<\/span><span class=\"w\"> <\/span><span class=\"s2\">\";C:\\Users\\user\\AppData\\Local\\Programs\\oh-my-posh\\bin\"<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Next, need to configure shell to use oh-my-posh. Edit your PowerShell profile script, you can find its location under the <code class=\"language-plaintext highlighter-rouge\">$PROFILE<\/code> variable in your preferred PowerShell version:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"err\">&lt;<\/span><span class=\"n\">editor<\/span><span class=\"err\">&gt;<\/span><span class=\"w\"> <\/span><span class=\"bp\">$PROFILE<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>If it returns an error. It probably means that the profile is not created yet. To fix this, run the following command:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">New-Item<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Path<\/span><span class=\"w\"> <\/span><span class=\"bp\">$PROFILE<\/span><span class=\"w\"> <\/span><span class=\"nt\">-ItemType<\/span><span class=\"w\"> <\/span><span class=\"nx\">File<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Force<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Add the following snippet as the last line to your PowerShell profile script:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">oh-my-posh<\/span><span class=\"w\"> <\/span><span class=\"nx\">init<\/span><span class=\"w\"> <\/span><span class=\"nx\">pwsh<\/span><span class=\"w\"> <\/span><span class=\"o\">|<\/span><span class=\"w\"> <\/span><span class=\"n\">Invoke-Expression<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Restart your terminal or reload your PowerShell profile to apply the changes:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"o\">.<\/span><span class=\"w\"> <\/span><span class=\"bp\">$PROFILE<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Lastly, choose the theme you like. I use the <code class=\"language-plaintext highlighter-rouge\">paradox<\/code> theme. You can find more themes and instructions on the <a href=\"https:\/\/ohmyposh.dev\/docs\/themes\">Oh My Posh Themes<\/a>. For me, I chose the <code class=\"language-plaintext highlighter-rouge\">powerlevel10k<\/code> theme. Edit your <code class=\"language-plaintext highlighter-rouge\">$PROFILE<\/code> file and add the following line:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># ---- Init Posh Theme ----<\/span><span class=\"w\">\n<\/span><span class=\"n\">oh-my-posh<\/span><span class=\"w\"> <\/span><span class=\"nx\">init<\/span><span class=\"w\"> <\/span><span class=\"nx\">pwsh<\/span><span class=\"w\"> <\/span><span class=\"nt\">--config<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"<\/span><span class=\"nv\">${env:POSH_THEMES_PATH}<\/span><span class=\"s2\">\\powerlevel10k_rainbow.omp.json\"<\/span><span class=\"w\"> <\/span><span class=\"o\">|<\/span><span class=\"w\"> <\/span><span class=\"n\">Invoke-Expression<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<h2 id=\"terminal-icons\">terminal icons<\/h2>\n\n<p>This is for better <code class=\"language-plaintext highlighter-rouge\">Get-ChildItem<\/code> results, which shows icons. Refer to <a href=\"github.com\/devblackops\/Terminal-Icons?tab=readme-ov-file#installations\">Terminal-Icons GitHub Page<\/a>. To install the module from the PowerShell Gallery:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">Install-Module<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">Terminal-Icons<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Repository<\/span><span class=\"w\"> <\/span><span class=\"nx\">PSGallery<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Configure the module by adding the following line to your <code class=\"language-plaintext highlighter-rouge\">$PROFILE<\/code> file:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># ---- Better Dir List ----<\/span><span class=\"w\">\n<\/span><span class=\"n\">Import-Module<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">Terminal-Icons<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<h2 id=\"completionpredictor\">CompletionPredictor<\/h2>\n\n<p>This is for better <code class=\"language-plaintext highlighter-rouge\">Tab<\/code> completion. Refer to <a href=\"https:\/\/github.com\/PowerShell\/CompletionPredictor?tab=readme-ov-file#use-the-predictor\">CompletionPredictor GitHub Page<\/a>. The CompletionPredictor plugin is built on the Subsystem Plugin Model, which is available with PowerShell 7.2 or above. To display prediction suggestions from the CompletionPredictor, you need PSReadLine 2.2.2 or above.<\/p>\n\n<ul>\n  <li><a href=\"https:\/\/learn.microsoft.com\/en-us\/powershell\/scripting\/learn\/experimental-features?view=powershell-7.5#pssubsystempluginmodel\">PowerShell 7.2 or above<\/a><\/li>\n  <li><a href=\"https:\/\/www.powershellgallery.com\/packages\/PSReadLine\/2.2.2\">PSReadLine 2.2.2 or above<\/a><\/li>\n<\/ul>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># Enable PSSubsystemPluginModel, required restart<\/span><span class=\"w\">\n<\/span><span class=\"n\">Enable-ExperimentalFeature<\/span><span class=\"w\"> <\/span><span class=\"nx\">PSSubsystemPluginModel<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># Install PSReadLine<\/span><span class=\"w\">\n<\/span><span class=\"n\">Install-Module<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">PSReadLine<\/span><span class=\"w\"> <\/span><span class=\"nt\">-RequiredVersion<\/span><span class=\"w\"> <\/span><span class=\"nx\">2.2.2<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># Install CompletionPredictor<\/span><span class=\"w\">\n<\/span><span class=\"n\">Install-Module<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">CompletionPredictor<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Repository<\/span><span class=\"w\"> <\/span><span class=\"nx\">PSGallery<\/span><span class=\"w\">\n<\/span><span class=\"n\">Import-Module<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">CompletionPredictor<\/span><span class=\"w\">\n<\/span><span class=\"n\">Set-PSReadLineOption<\/span><span class=\"w\"> <\/span><span class=\"nt\">-PredictionSource<\/span><span class=\"w\"> <\/span><span class=\"nx\">HistoryAndPlugin<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>For toggling between inline and list mode, press <code class=\"language-plaintext highlighter-rouge\">F2<\/code>. I configured it in my <code class=\"language-plaintext highlighter-rouge\">$PROFILE<\/code> file like:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># ---- Better Prediction ----<\/span><span class=\"w\">\n<\/span><span class=\"n\">Set-PSReadLineOption<\/span><span class=\"w\"> <\/span><span class=\"nt\">-PredictionViewStyle<\/span><span class=\"w\"> <\/span><span class=\"nx\">ListView<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p><br \/><\/p>\n<figure class=\"image-card width-wide caption\">\n  <img src=\"https:\/\/github.com\/PowerShell\/CompletionPredictor\/raw\/main\/tools\/images\/CompletionPredictor.gif\" alt=\"CompletionPredictor\" \/>\n  \n  <figcaption class=\"caption-text\">A screenshot of using CompletionPredictor, from https:\/\/github.com\/PowerShell\/CompletionPredictor<\/figcaption>\n  \n<\/figure>\n\n<hr \/>\n\n<h1 id=\"command-line-tools\">Command-Line Tools<\/h1>\n\n<h2 id=\"tldr---community-driven-man\">tldr - Community-driven man<\/h2>\n\n<p>A much more user-friendly man command. You can find more information and installation instructions on the <a href=\"https:\/\/github.com\/tldr-pages\/tldr\">tldr GitHub Page<\/a>. Installation via Chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">tldr<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>It can show you the command example, usage, and description. And even some <strong>initialization settings<\/strong>. (I actually use this tool a lot. And for zoxide initialization, I just copy and paste given suggestions into my <code class=\"language-plaintext highlighter-rouge\">$PROFILE<\/code>.)<\/p>\n\n<h2 id=\"bat---better-cat\">bat - Better cat<\/h2>\n\n<p>Basically, this is a better <code class=\"language-plaintext highlighter-rouge\">cat<\/code> command with syntax highlighting and user-friendly pagenation. Check out <a href=\"https:\/\/github.com\/sharkdp\/bat\">bat GitHub Page<\/a>. Installation via Chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">bat<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<h2 id=\"winfetch---system-info\">winfetch - System Info<\/h2>\n\n<p>A fancy tool to show system info. Check out <a href=\"https:\/\/github.com\/lptstr\/winfetch\">winfetch GitHub Page<\/a>. Installation via Chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">winfetch<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>See my cover of this post. It can even show images in the terminal! I show my avatar on Github~<\/p>\n\n<h2 id=\"dust---quick-du\">dust - Quick du<\/h2>\n\n<p>Quicker du replacement. It is written in Rust. Check out <a href=\"https:\/\/github.com\/bootandy\/dust\">dust GitHub Page<\/a>. Installation via Chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">dust<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>gdu is another quick du replacement. It is written in Go. Check out <a href=\"https:\/\/github.com\/dundee\/gdu\">gdu GitHub Page<\/a>. Installation via Chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">gdu<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>(actually, I prefer gdu\u2019s ui over dust\u2019s ui)<\/p>\n\n<h2 id=\"bottom---better-top\">bottom - Better top<\/h2>\n\n<p>Opposite of top. Just joking. Check out <a href=\"https:\/\/github.com\/ClementTsang\/bottom\">bottom GitHub Page<\/a>. Installation via Chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">bottom<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>It is also a dependency of astronvim.<\/p>\n\n<h2 id=\"fzf---fuzzy-finder\">fzf - Fuzzy finder<\/h2>\n\n<p>Greatness needs no words! Check out <a href=\"https:\/\/github.com\/junegunn\/fzf\">fzf GitHub Page<\/a>. Installation via Chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">fzf<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<h2 id=\"zoxide---smart-cd\">zoxide - Smart cd<\/h2>\n\n<p>It will change your terminal change path cmd from <code class=\"language-plaintext highlighter-rouge\">cd<\/code> to <code class=\"language-plaintext highlighter-rouge\">z<\/code>. Check out <a href=\"https:\/\/github.com\/ajeetdsouza\/zoxide\">zoxide GitHub Page<\/a>. Installation via Chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">zoxide<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<h2 id=\"yazi---tui-file-manager\">yazi - TUI file manager<\/h2>\n\n<p>This is a blazing fast terminal file manager written in Rust. Check out <a href=\"https:\/\/yazi-rs.github.io\/\">Yazi Official Docs<\/a> or the <a href=\"https:\/\/github.com\/sxyazi\/yazi\">Yazi GitHub Page<\/a>. Installation via Chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">yazi<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>It is a very nice match for terminal-based workflows, especially if you already use tools like <code class=\"language-plaintext highlighter-rouge\">fzf<\/code>, <code class=\"language-plaintext highlighter-rouge\">zoxide<\/code>, <code class=\"language-plaintext highlighter-rouge\">bat<\/code>, and <code class=\"language-plaintext highlighter-rouge\">ripgrep<\/code>.<\/p>\n\n<p>One important note for Windows users: <code class=\"language-plaintext highlighter-rouge\">yazi<\/code> relies on the <code class=\"language-plaintext highlighter-rouge\">file<\/code> tool to detect file mime types, and unlike Unix-like systems, Windows does not provide it by default. The community-recommended solution is to reuse the <code class=\"language-plaintext highlighter-rouge\">file.exe<\/code> shipped with Git for Windows:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">setx<\/span><span class=\"w\"> <\/span><span class=\"nx\">YAZI_FILE_ONE<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"C:\\Program Files\\Git\\usr\\bin\\file.exe\"<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>After that, restart your terminal. This is also the recommended setup in the official Yazi docs. If you installed Git in a different location, change the path accordingly.<\/p>\n\n<hr \/>\n\n<h1 id=\"development-settings\">Development Settings<\/h1>\n\n<p>In this section, I will introduce some useful tui tools for development.<\/p>\n\n<h2 id=\"lazygit---tui-for-git-commands\">lazygit - TUI for git commands<\/h2>\n\n<p>This <a href=\"https:\/\/github.com\/jesseduffield\/lazygit\">open source<\/a> tui tool just boost my git workflow. The most magic part is that it can commit patch by patch! You can find more information and installation instructions on the <a href=\"https:\/\/jesseduffield.com\/Lazygit-5-Years-On\/\">LazyGit Post<\/a>. Installation via Chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">lazygit<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p><br \/><\/p>\n<figure class=\"image-card width-wide caption\">\n  <img src=\"https:\/\/github.com\/jesseduffield\/lazygit\/raw\/assets\/demo\/commit_and_push-compressed.gif\" alt=\"LazyGit\" \/>\n  \n  <figcaption class=\"caption-text\">A screenshot of using lazygit, from https:\/\/github.com\/jesseduffield\/lazygit<\/figcaption>\n  \n<\/figure>\n\n<h2 id=\"lazydocker---tui-for-docker-commands\">lazydocker - TUI for docker commands<\/h2>\n\n<p>Another powerful <a href=\"https:\/\/github.com\/jesseduffield\/lazydocker\">open source<\/a> tui tool. It will level up your docker experience. The author is <a href=\"https:\/\/github.com\/jesseduffield\">jesseduffield<\/a>, the same author of <a href=\"https:\/\/github.com\/jesseduffield\/lazygit\">lazygit<\/a>. Installation via Chocolatey:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">lazydocker<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p><br \/><\/p>\n<figure class=\"image-card width-wide caption\">\n  <img src=\"https:\/\/github.com\/jesseduffield\/lazydocker\/raw\/master\/docs\/resources\/demo3.gif\" alt=\"LazyDocker\" \/>\n  \n  <figcaption class=\"caption-text\">A screenshot of using lazydocker, from https:\/\/github.com\/jesseduffield\/lazydocker<\/figcaption>\n  \n<\/figure>\n\n<h2 id=\"astronvim---distr-of-neovim\">astronvim - Distr of Neovim<\/h2>\n\n<p>This is my favorite Neovim distribution. It comes with a lot of pre-configured plugins and settings that make it easy to use right out of the box. You can find more information and installation instructions on the <a href=\"https:\/\/github.com\/AstroNvim\/AstroNvim\">Astronvim GitHub page<\/a>, or check the official website at <a href=\"https:\/\/astronvim.com\/\">https:\/\/astronvim.com\/<\/a>. Before configuring Astronvim, make sure you have Neovim installed. You can install Neovim via Chocolatey with the command:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">install<\/span><span class=\"w\"> <\/span><span class=\"nx\">neovim<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>And also we have some dependencies for better coding experience:<\/p>\n\n<ul>\n  <li>Nerd Fonts: done at <a href=\"#font\">font<\/a>. Necessary for proper display of icons in the status line and file explorer.<\/li>\n  <li>ripgrep: A fast search tool that is used by many Neovim plugins for searching files. Install via Chocolatey: <code class=\"language-plaintext highlighter-rouge\">choco install ripgrep<\/code>. <code class=\"language-plaintext highlighter-rouge\">&lt;Leader&gt;fw<\/code> to search files.<\/li>\n  <li>lazygit: done at <a href=\"#lazygit\">lazygit<\/a>. Default git client for Astronvim. Install via Chocolatey: <code class=\"language-plaintext highlighter-rouge\">choco install lazygit<\/code>. <code class=\"language-plaintext highlighter-rouge\">&lt;Leader&gt;gg<\/code> to open lazygit.<\/li>\n  <li>gdu: Short for \u201cgo disk usage\u201d, A fast disk usage analyzer that can be used within Neovim. Install via Chocolatey: <code class=\"language-plaintext highlighter-rouge\">choco install gdu<\/code>. <code class=\"language-plaintext highlighter-rouge\">&lt;Leader&gt;gs<\/code> to open gdu.<\/li>\n  <li>bottom: done at <a href=\"#bottom\">bottom<\/a>. A better performance monitor. Install via Chocolatey: <code class=\"language-plaintext highlighter-rouge\">choco install bottom<\/code>. <code class=\"language-plaintext highlighter-rouge\">&lt;Leader&gt;tt<\/code> to open bottom.<\/li>\n  <li>Python: Install via Chocolatey: <code class=\"language-plaintext highlighter-rouge\">choco install python<\/code>. <code class=\"language-plaintext highlighter-rouge\">&lt;Leader&gt;tp<\/code> to open Python REPL.<\/li>\n  <li>Node.js: Node is needed for a lot of the LSPs. Install via Chocolatey: <code class=\"language-plaintext highlighter-rouge\">choco install nodejs<\/code>. <code class=\"language-plaintext highlighter-rouge\">&lt;Leader&gt;tn<\/code> to open Node.js REPL.<\/li>\n<\/ul>\n\n<p>Next, just follow the <a href=\"https:\/\/docs.astronvim.com\/#-installation\">Astronvim Installation Guide<\/a> to set it up.<\/p>\n\n<ul>\n  <li>\n    <p>Make a backup of your current nvim config (if exists)<\/p>\n\n    <div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"w\">  <\/span><span class=\"n\">Move-Item<\/span><span class=\"w\"> <\/span><span class=\"nv\">$<\/span><span class=\"nn\">env<\/span><span class=\"p\">:<\/span><span class=\"nv\">LOCALAPPDATA<\/span><span class=\"nx\">\\nvim<\/span><span class=\"w\"> <\/span><span class=\"nv\">$<\/span><span class=\"nn\">env<\/span><span class=\"p\">:<\/span><span class=\"nv\">LOCALAPPDATA<\/span><span class=\"nx\">\\nvim.bak<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div>    <\/div>\n  <\/li>\n  <li>\n    <p>Clean neovim folders (Optional but recommended)<\/p>\n\n    <div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"w\">  <\/span><span class=\"n\">Move-Item<\/span><span class=\"w\"> <\/span><span class=\"nv\">$<\/span><span class=\"nn\">env<\/span><span class=\"p\">:<\/span><span class=\"nv\">LOCALAPPDATA<\/span><span class=\"nx\">\\nvim-data<\/span><span class=\"w\"> <\/span><span class=\"nv\">$<\/span><span class=\"nn\">env<\/span><span class=\"p\">:<\/span><span class=\"nv\">LOCALAPPDATA<\/span><span class=\"nx\">\\nvim-data.bak<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div>    <\/div>\n  <\/li>\n  <li>\n    <p>Clone the repository<\/p>\n\n    <div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"w\">  <\/span><span class=\"n\">git<\/span><span class=\"w\"> <\/span><span class=\"nx\">clone<\/span><span class=\"w\"> <\/span><span class=\"nt\">--depth<\/span><span class=\"w\"> <\/span><span class=\"nx\">1<\/span><span class=\"w\"> <\/span><span class=\"nx\">https:\/\/github.com\/AstroNvim\/template<\/span><span class=\"w\"> <\/span><span class=\"nv\">$<\/span><span class=\"nn\">env<\/span><span class=\"p\">:<\/span><span class=\"nv\">LOCALAPPDATA<\/span><span class=\"nx\">\\nvim<\/span><span class=\"w\">\n  <\/span><span class=\"c\"># remove template's git connection to set up your own later<\/span><span class=\"w\">\n  <\/span><span class=\"n\">Remove-Item<\/span><span class=\"w\"> <\/span><span class=\"nv\">$<\/span><span class=\"nn\">env<\/span><span class=\"p\">:<\/span><span class=\"nv\">LOCALAPPDATA<\/span><span class=\"nx\">\\nvim\\.git<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Recurse<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Force<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div>    <\/div>\n  <\/li>\n<\/ul>\n\n<p>In another post of mine, I also talked about how to set up <code class=\"language-plaintext highlighter-rouge\">windsurf<\/code> (powerful but free AI tool) with <code class=\"language-plaintext highlighter-rouge\">Neovim<\/code>. please check it out <a href=\"\/Linux-Terminal-Beautify#integrate-with-ai-tools\">here<\/a>.<\/p>\n\n<h2 id=\"openssh-server-and-client\">OpenSSH Server and Client<\/h2>\n\n<p>Setting up SSH is a little bit different on Windows compared to Unix-like systems, see <a href=\"https:\/\/learn.microsoft.com\/en-us\/windows-server\/administration\/openssh\/openssh_install_firstuse?tabs=powershell&amp;pivots=windows-11\">OpenSSH Installation Guide<\/a>. But still a easy task. First, check if you have administrator privileges:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"p\">(<\/span><span class=\"n\">New-Object<\/span><span class=\"w\"> <\/span><span class=\"nx\">Security.Principal.WindowsPrincipal<\/span><span class=\"p\">([<\/span><span class=\"n\">Security.Principal.WindowsIdentity<\/span><span class=\"p\">]::<\/span><span class=\"n\">GetCurrent<\/span><span class=\"p\">()))<\/span><span class=\"o\">.<\/span><span class=\"nf\">IsInRole<\/span><span class=\"p\">([<\/span><span class=\"n\">Security.Principal.WindowsBuiltInRole<\/span><span class=\"p\">]::<\/span><span class=\"nx\">Administrator<\/span><span class=\"p\">)<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>If got <code class=\"language-plaintext highlighter-rouge\">True<\/code>, you can just run the following command to check if OpenSSH is installed. Remember to run it with administrator privileges:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">Get-WindowsCapability<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Online<\/span><span class=\"w\"> <\/span><span class=\"o\">|<\/span><span class=\"w\"> <\/span><span class=\"n\">Where-Object<\/span><span class=\"w\"> <\/span><span class=\"nx\">Name<\/span><span class=\"w\"> <\/span><span class=\"o\">-like<\/span><span class=\"w\"> <\/span><span class=\"s1\">'OpenSSH*'<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>If got something like this:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">Name<\/span><span class=\"w\">  <\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"nx\">OpenSSH.Client~~~~0.0.1.0<\/span><span class=\"w\">\n<\/span><span class=\"n\">State<\/span><span class=\"w\"> <\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"nx\">NotPresent<\/span><span class=\"w\">\n\n<\/span><span class=\"n\">Name<\/span><span class=\"w\">  <\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"nx\">OpenSSH.Server~~~~0.0.1.0<\/span><span class=\"w\">\n<\/span><span class=\"n\">State<\/span><span class=\"w\"> <\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"nx\">NotPresent<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>We need to install through following command:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># Install the OpenSSH Client<\/span><span class=\"w\">\n<\/span><span class=\"n\">Add-WindowsCapability<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Online<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">OpenSSH.Client~~~~0.0.1.0<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># Install the OpenSSH Server<\/span><span class=\"w\">\n<\/span><span class=\"n\">Add-WindowsCapability<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Online<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">OpenSSH.Server~~~~0.0.1.0<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>It will take a few minutes to install. Once done, run the following commands to start the sshd service:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># Start the sshd service<\/span><span class=\"w\">\n<\/span><span class=\"n\">Start-Service<\/span><span class=\"w\"> <\/span><span class=\"nx\">sshd<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># OPTIONAL but recommended:<\/span><span class=\"w\">\n<\/span><span class=\"n\">Set-Service<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">sshd<\/span><span class=\"w\"> <\/span><span class=\"nt\">-StartupType<\/span><span class=\"w\"> <\/span><span class=\"s1\">'Automatic'<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># Confirm the Firewall rule is configured. It should be created automatically by setup. Run the following to verify<\/span><span class=\"w\">\n<\/span><span class=\"kr\">if<\/span><span class=\"w\"> <\/span><span class=\"p\">(<\/span><span class=\"o\">!<\/span><span class=\"p\">(<\/span><span class=\"n\">Get-NetFirewallRule<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"OpenSSH-Server-In-TCP\"<\/span><span class=\"w\"> <\/span><span class=\"nt\">-ErrorAction<\/span><span class=\"w\"> <\/span><span class=\"nx\">SilentlyContinue<\/span><span class=\"p\">))<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"n\">Write-Output<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Firewall Rule 'OpenSSH-Server-In-TCP' does not exist, creating it...\"<\/span><span class=\"w\">\n    <\/span><span class=\"n\">New-NetFirewallRule<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"s1\">'OpenSSH-Server-In-TCP'<\/span><span class=\"w\"> <\/span><span class=\"nt\">-DisplayName<\/span><span class=\"w\"> <\/span><span class=\"s1\">'OpenSSH Server (sshd)'<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Enabled<\/span><span class=\"w\"> <\/span><span class=\"nx\">True<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Direction<\/span><span class=\"w\"> <\/span><span class=\"nx\">Inbound<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Protocol<\/span><span class=\"w\"> <\/span><span class=\"nx\">TCP<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Action<\/span><span class=\"w\"> <\/span><span class=\"nx\">Allow<\/span><span class=\"w\"> <\/span><span class=\"nt\">-LocalPort<\/span><span class=\"w\"> <\/span><span class=\"nx\">22<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\"> <\/span><span class=\"kr\">else<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"n\">Write-Output<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"Firewall rule 'OpenSSH-Server-In-TCP' has been created and exists.\"<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Next, we need to set up SSH keys. It is different for administrators and non-administrators. For non-administrators, we can add public keys to the <code class=\"language-plaintext highlighter-rouge\">authorized_keys<\/code> file of the user.<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">nvim<\/span><span class=\"w\"> <\/span><span class=\"nx\">~\\.ssh\\authorized_keys<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>For administrators, add public keys to the <code class=\"language-plaintext highlighter-rouge\">administrators_authorized_keys<\/code> file which located at <code class=\"language-plaintext highlighter-rouge\">C:\\ProgramData\\ssh\\<\/code>:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">nvim<\/span><span class=\"w\"> <\/span><span class=\"nx\">C:\\ProgramData\\ssh\\administrators_authorized_keys<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Initially, OpenSSH server will use <code class=\"language-plaintext highlighter-rouge\">cmd.exe<\/code> as the default shell. Let\u2019s change it to <code class=\"language-plaintext highlighter-rouge\">powershell.exe<\/code>:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"nv\">$NewItemPropertyParams<\/span><span class=\"w\"> <\/span><span class=\"o\">=<\/span><span class=\"w\"> <\/span><span class=\"p\">@{<\/span><span class=\"w\">\n    <\/span><span class=\"nx\">Path<\/span><span class=\"w\">         <\/span><span class=\"o\">=<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"HKLM:\\SOFTWARE\\OpenSSH\"<\/span><span class=\"w\">\n    <\/span><span class=\"nx\">Name<\/span><span class=\"w\">         <\/span><span class=\"o\">=<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"DefaultShell\"<\/span><span class=\"w\">\n    <\/span><span class=\"nx\">Value<\/span><span class=\"w\">        <\/span><span class=\"o\">=<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"C:\\Program Files\\PowerShell\\7\\pwsh.exe\"<\/span><span class=\"w\"> <\/span><span class=\"c\"># or change to the path of favorite shell executable<\/span><span class=\"w\">\n    <\/span><span class=\"nx\">PropertyType<\/span><span class=\"w\"> <\/span><span class=\"o\">=<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"String\"<\/span><span class=\"w\">\n    <\/span><span class=\"nx\">Force<\/span><span class=\"w\">        <\/span><span class=\"o\">=<\/span><span class=\"w\"> <\/span><span class=\"bp\">$true<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"n\">New-ItemProperty<\/span><span class=\"w\"> <\/span><span class=\"err\">@<\/span><span class=\"nx\">NewItemPropertyParams<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>Boom! We are now ready to use SSH to connect to the Windows machine!<\/p>\n\n<hr \/>\n\n<h1 id=\"conclusion\">Conclusion<\/h1>\n\n<p>Customize your terminal to fit your workflow and enjoy a more efficient coding experience on Windows! And here is my final checklist of tools and settings (I deleted some less relevant tools for brevity):<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># Chocolatey Packages<\/span><span class=\"w\">\n<\/span><span class=\"err\">&gt;<\/span><span class=\"w\"> <\/span><span class=\"n\">choco<\/span><span class=\"w\"> <\/span><span class=\"nx\">list<\/span><span class=\"w\">\n<\/span><span class=\"n\">Chocolatey<\/span><span class=\"w\"> <\/span><span class=\"nx\">v2.5.1<\/span><span class=\"w\">\n<\/span><span class=\"n\">bat<\/span><span class=\"w\"> <\/span><span class=\"nx\">0.25.0<\/span><span class=\"w\">\n<\/span><span class=\"n\">bottom<\/span><span class=\"w\"> <\/span><span class=\"nx\">0.11.2<\/span><span class=\"w\">\n<\/span><span class=\"n\">chocolatey<\/span><span class=\"w\"> <\/span><span class=\"nx\">2.5.1<\/span><span class=\"w\">\n<\/span><span class=\"n\">dust<\/span><span class=\"w\"> <\/span><span class=\"nx\">1.2.3<\/span><span class=\"w\">\n<\/span><span class=\"n\">fastfetch<\/span><span class=\"w\"> <\/span><span class=\"nx\">2.53.0<\/span><span class=\"w\">\n<\/span><span class=\"n\">fzf<\/span><span class=\"w\"> <\/span><span class=\"nx\">0.59.0<\/span><span class=\"w\">\n<\/span><span class=\"n\">git<\/span><span class=\"w\"> <\/span><span class=\"nx\">2.51.0.2<\/span><span class=\"w\">\n<\/span><span class=\"n\">git.install<\/span><span class=\"w\"> <\/span><span class=\"nx\">2.51.0.2<\/span><span class=\"w\">\n<\/span><span class=\"n\">Lazydocker<\/span><span class=\"w\"> <\/span><span class=\"nx\">0.23.3<\/span><span class=\"w\">\n<\/span><span class=\"n\">lazygit<\/span><span class=\"w\"> <\/span><span class=\"nx\">0.55.1<\/span><span class=\"w\">\n<\/span><span class=\"n\">less<\/span><span class=\"w\"> <\/span><span class=\"nx\">679.0.0<\/span><span class=\"w\">\n<\/span><span class=\"n\">neovim<\/span><span class=\"w\"> <\/span><span class=\"nx\">0.11.4<\/span><span class=\"w\">\n<\/span><span class=\"n\">nodejs<\/span><span class=\"w\"> <\/span><span class=\"nx\">22.20.0<\/span><span class=\"w\">\n<\/span><span class=\"n\">nodejs.install<\/span><span class=\"w\"> <\/span><span class=\"nx\">22.20.0<\/span><span class=\"w\">\n<\/span><span class=\"n\">oh-my-posh<\/span><span class=\"w\"> <\/span><span class=\"nx\">27.1.2<\/span><span class=\"w\">\n<\/span><span class=\"n\">ripgrep<\/span><span class=\"w\"> <\/span><span class=\"nx\">14.1.0<\/span><span class=\"w\">\n<\/span><span class=\"n\">tldr<\/span><span class=\"w\"> <\/span><span class=\"nx\">1.0.0<\/span><span class=\"w\">\n<\/span><span class=\"n\">tldr-plusplus<\/span><span class=\"w\"> <\/span><span class=\"nx\">0.6.1<\/span><span class=\"w\">\n<\/span><span class=\"n\">winfetch<\/span><span class=\"w\"> <\/span><span class=\"nx\">2.5.1<\/span><span class=\"w\">\n<\/span><span class=\"n\">yazi<\/span><span class=\"w\"> <\/span><span class=\"nx\">26.1.22<\/span><span class=\"w\">\n<\/span><span class=\"n\">zoxide<\/span><span class=\"w\"> <\/span><span class=\"nx\">0.9.2<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>And here is my current profile settings:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\"># ---- Oh My Posh ----<\/span><span class=\"w\">\n<\/span><span class=\"n\">oh-my-posh<\/span><span class=\"w\"> <\/span><span class=\"nx\">init<\/span><span class=\"w\"> <\/span><span class=\"nx\">pwsh<\/span><span class=\"w\"> <\/span><span class=\"nt\">--config<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"<\/span><span class=\"nv\">${env:POSH_THEMES_PATH}<\/span><span class=\"s2\">\\powerlevel10k_rainbow.omp.json\"<\/span><span class=\"w\"> <\/span><span class=\"o\">|<\/span><span class=\"w\"> <\/span><span class=\"n\">Invoke-Expression<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># ---- Terminal Icons ----<\/span><span class=\"w\">\n<\/span><span class=\"n\">Import-Module<\/span><span class=\"w\"> <\/span><span class=\"nt\">-Name<\/span><span class=\"w\"> <\/span><span class=\"nx\">Terminal-Icons<\/span><span class=\"w\">\n  \n<\/span><span class=\"c\"># ---- PSReadLine ----<\/span><span class=\"w\">\n<\/span><span class=\"n\">Set-PSReadLineOption<\/span><span class=\"w\"> <\/span><span class=\"nt\">-PredictionViewStyle<\/span><span class=\"w\"> <\/span><span class=\"nx\">ListView<\/span><span class=\"w\">\n\n<\/span><span class=\"c\"># ---- Zoxide ----<\/span><span class=\"w\">\n<\/span><span class=\"c\"># Just copy and pasted from tldr zoxide init instructions. It works! :)<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>If you also use <code class=\"language-plaintext highlighter-rouge\">yazi<\/code>, remember that Windows needs an extra environment variable for <code class=\"language-plaintext highlighter-rouge\">file.exe<\/code> support:<\/p>\n\n<div class=\"language-powershell highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">setx<\/span><span class=\"w\"> <\/span><span class=\"nx\">YAZI_FILE_ONE<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"C:\\Program Files\\Git\\usr\\bin\\file.exe\"<\/span><span class=\"w\">\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p>For Powershell Modules and PSSusbsytems, refer to the <a href=\"#powershell-modules-and-pssubsystems\">Powershell Modules and PSSubsystems Checklist<\/a>.<\/p>\n\n<hr \/>\n\n<p>Related Posts \/ Websites \ud83d\udc47<\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/timdeschryver.dev\/bits\/creating-a-good-looking-windows-terminal\">Tim Deschryver - Creating A Good Looking Windows Terminal<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/timdeschryver.dev\/blog\/how-i-have-set-up-my-new-windows-development-environment-in-2022\">Tim Deschryver - How I Have Set Up My New Windows Development Environment in 2022<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/learn.microsoft.com\/en-us\/windows\/terminal\/tips-and-tricks?WT.mc_id=DT-MVP-5004452#quake-mode\">Microsoft - Quake Mode<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/learn.microsoft.com\/en-us\/powershell\/scripting\/install\/installing-powershell-on-windows?view=powershell-7.5\">Microsoft - Installing Powershell on Windows<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/learn.microsoft.com\/en-us\/powershell\/module\/microsoft.powershell.core\/get-pssubsystem?view=powershell-7.5\">Microsoft - Get PSSsubsystem<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/learn.microsoft.com\/en-us\/powershell\/module\/microsoft.powershell.core\/get-module?view=powershell-7.5\">Microsoft - Get-Module<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/learn.microsoft.com\/en-us\/windows-server\/administration\/openssh\/openssh_install_firstuse?tabs=powershell&amp;pivots=windows-11\">Microsoft - OpenSSH Installation Guide<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/learn.microsoft.com\/en-us\/windows-server\/administration\/openssh\/openssh_keymanagement\">Microsoft - OpenSSH Key Management<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/github.com\/microsoft\/winget-cli\">GitHub - winget<\/a>, <a href=\"https:\/\/github.com\/chocolatey\/choco\">GitHub - chocolatey<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/github.com\/JanDeDobbeleer\/oh-my-posh\">GitHub - oh-my-posh<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/github.com\/tldr-pages\/tldr\">GitHub - tldr<\/a>, <a href=\"https:\/\/github.com\/sharkdp\/bat\">GitHub - bat<\/a>, <a href=\"https:\/\/github.com\/bootandy\/dust\">GitHub - dust<\/a>, <a href=\"https:\/\/github.com\/dundee\/gdu\">GitHub - gdu<\/a>, <a href=\"https:\/\/github.com\/junegunn\/fzf\">GitHub - fzf<\/a>, <a href=\"https:\/\/github.com\/ajeetdsouza\/zoxide\">GitHub - zoxide<\/a>, <a href=\"https:\/\/github.com\/sxyazi\/yazi\">GitHub - yazi<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/github.com\/jesseduffield\/lazygit\">GitHub - lazygit<\/a>, <a href=\"https:\/\/github.com\/jesseduffield\/lazydocker\">GitHub - lazydocker<\/a>, <a href=\"https:\/\/github.com\/neovim\/neovim\">GitHub - neovim<\/a><\/p>\n","pubDate":"Thu, 16 Oct 2025 00:00:00 +0000","link":"https:\/\/huruilizhen.github.io\/windows-production-tools","guid":"https:\/\/huruilizhen.github.io\/windows-production-tools","category":["windows","configuration","command-line-interface"]},{"title":"Technical Classification and Implementation of Hotkeys in Windows Desktop Applications","description":"<p>Hotkeys are indispensable for enhancing user experience in Windows desktop applications. They enable users to quickly access features, trigger actions, or interact with specific controls using keyboard combinations. In this post, we examine the technical classification of hotkeys within the Windows environment and explore the underlying implementation mechanisms, with a comparative look at how native Win32 APIs and the Qt framework handle these operations.<\/p>\n\n<!--more-->\n\n<h2 id=\"table-of-contents\">Table of Contents<\/h2>\n<ul>\n  <li><a href=\"#types-of-hotkeys-in-windows-applications\">Types of Hotkeys in Windows Applications<\/a><\/li>\n  <li><a href=\"#implementation-principle-of-shortcut-key-technology\">Implementation principle of shortcut key technology<\/a>\n    <ul>\n      <li><a href=\"#window-states-and-their-roles-in-hotkey-handling\">Window States and Their Roles in Hotkey Handling<\/a><\/li>\n      <li><a href=\"#message-queue-in-native-windows-desktop-applications\">Message Queue in Native Windows Desktop Applications<\/a>\n        <ul>\n          <li><a href=\"#message-sources-and-structure\">Message Sources and Structure<\/a><\/li>\n          <li><a href=\"#workflow-of-the-message-queue\">Workflow of the Message Queue<\/a><\/li>\n          <li><a href=\"#relationship-between-message-queue-and-keyboard-shortcuts\">Relationship Between Message Queue and Keyboard Shortcuts<\/a><\/li>\n        <\/ul>\n      <\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#comparing-native-win32-and-qt-implementations\">Comparing Native Win32 and Qt Implementations<\/a>\n    <ul>\n      <li><a href=\"#win32-api\">Win32 API<\/a><\/li>\n      <li><a href=\"#qt-framework\">Qt Framework<\/a>\n        <ul>\n          <li><a href=\"#qt-message-mechanism-abstraction\">Qt Message Mechanism Abstraction<\/a><\/li>\n          <li><a href=\"#window-state-management-in-qt\">Window State Management in Qt<\/a><\/li>\n          <li><a href=\"#qt-shortcut-key-handling\">Qt Shortcut Key Handling<\/a><\/li>\n        <\/ul>\n      <\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#platform-specific-considerations-in-qt\">Platform-Specific Considerations in Qt<\/a><\/li>\n<\/ul>\n\n<hr \/>\n\n<h1 id=\"types-of-hotkeys-in-windows-applications\">Types of Hotkeys in Windows Applications<\/h1>\n\n<p>Hotkeys in Windows applications can be broadly classified into three categories:<\/p>\n\n<ul>\n  <li>\n    <p><strong>Global Hotkeys (System-Level):<\/strong> These work regardless of which application is in the foreground. Applications register such hotkeys using the <code class=\"language-plaintext highlighter-rouge\">RegisterHotKey<\/code> API. They are ideal for functionalities like screenshot tools, app launching, or system-wide commands.<\/p>\n  <\/li>\n  <li>\n    <p><strong>Application-Level Hotkeys:<\/strong> These function only when the application is active and in focus. Managed via the message loop by intercepting messages like <code class=\"language-plaintext highlighter-rouge\">WM_KEYDOWN<\/code>, <code class=\"language-plaintext highlighter-rouge\">WM_SYSKEYDOWN<\/code>, and often associated with accelerator tables.<\/p>\n  <\/li>\n  <li>\n    <p><strong>Control-Level Hotkeys:<\/strong> Valid only when a particular UI control (e.g., a text box or button) is focused. The control itself handles the keyboard input internally (e.g., <code class=\"language-plaintext highlighter-rouge\">Ctrl+C<\/code>, <code class=\"language-plaintext highlighter-rouge\">Ctrl+V<\/code> for copy\/paste in text boxes).<\/p>\n  <\/li>\n<\/ul>\n\n<hr \/>\n\n<h1 id=\"implementation-principle-of-shortcut-key-technology\">Implementation principle of shortcut key technology<\/h1>\n\n<p>Understanding Windows desktop applications requires grasping two foundational concepts: window states and message queues. This section focuses on the former.<\/p>\n\n<h2 id=\"window-states-and-their-roles-in-hotkey-handling\">Window States and Their Roles in Hotkey Handling<\/h2>\n\n<p>Windows classifies window states into three main types:<\/p>\n\n<ul>\n  <li><strong>Foreground Window<\/strong>: This is the window currently interacting with the user and is considered the most \u201cimportant\u201d by the system. At any given time, only one window can hold the foreground status. The foreground window receives keyboard input and related messages. A window typically becomes foreground in one of two ways: either the user clicks on it or its taskbar icon, or a program explicitly calls SetForegroundWindow()\u2014though the latter is subject to permission restrictions. In practice, application-level shortcuts are only active when the window is in the foreground, as only the foreground window is eligible to receive keyboard input.<\/li>\n  <li><strong>Active Window<\/strong>: This refers to the window that is currently receiving input within a given thread. It is a subset of the foreground window but scoped to the current thread. While the foreground window is a system-wide concept, the active window is thread-specific. Application-level shortcuts are typically bound to the active window or its child widgets since only the active window is capable of handling keyboard messages in that context.<\/li>\n  <li><strong>Focus Window<\/strong>: The focus window is the exact component currently receiving keyboard input\u2014usually a child widget like a text box. It is a child of the active window. Widget-level shortcuts often rely on the focus window to intercept and process key events correctly.<\/li>\n<\/ul>\n\n<p>This layered structure\u2014foreground, active, and focus\u2014forms the basis for managing user interaction and keyboard event dispatch in native Windows applications.<\/p>\n\n<h2 id=\"message-queue-in-native-windows-desktop-applications\">Message Queue in Native Windows Desktop Applications<\/h2>\n\n<p>One of the core mechanisms underpinning Windows GUI programming is the message queue. A thorough understanding of it is crucial for mastering event-driven programming, shortcut handling, and window interactions. Windows operates on a message-driven model to handle user interface interactions. In essence:<\/p>\n<ul>\n  <li>User input (keyboard, mouse), system events (e.g., window resize, timer), and program-internal events are all encapsulated as messages.<\/li>\n  <li>These messages are placed into a message queue, awaiting processing by the application.<\/li>\n  <li>The application runs a message loop that continuously retrieves messages from the queue and dispatches them to the corresponding window procedure (WndProc) for handling.<\/li>\n<\/ul>\n\n<p>This event-driven architecture allows the application to respond reactively to both user actions and system-level changes.<\/p>\n\n<h3 id=\"message-sources-and-structure\">Message Sources and Structure<\/h3>\n\n<p>Messages originate from several sources:<\/p>\n<ul>\n  <li><strong>User input messages<\/strong>: e.g., keyboard keystrokes, mouse clicks, mouse movement.<\/li>\n  <li><strong>System messages<\/strong>: e.g., window creation and destruction, repaint requests, resizing, timer events.<\/li>\n  <li><strong>Application-defined messages<\/strong>: sent explicitly using APIs like <code class=\"language-plaintext highlighter-rouge\">PostMessage<\/code> or <code class=\"language-plaintext highlighter-rouge\">SendMessage<\/code>.<\/li>\n<\/ul>\n\n<p>Windows messages are represented by a <code class=\"language-plaintext highlighter-rouge\">MSG<\/code> structure:<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"k\">typedef<\/span> <span class=\"k\">struct<\/span> <span class=\"nc\">tagMSG<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">HWND<\/span>   <span class=\"n\">hwnd<\/span><span class=\"p\">;<\/span>      <span class=\"c1\">\/\/ Target window handle<\/span>\n    <span class=\"n\">UINT<\/span>   <span class=\"n\">message<\/span><span class=\"p\">;<\/span>   <span class=\"c1\">\/\/ Message type (e.g., WM_KEYDOWN)<\/span>\n    <span class=\"n\">WPARAM<\/span> <span class=\"n\">wParam<\/span><span class=\"p\">;<\/span>    <span class=\"c1\">\/\/ Additional info (context-specific)<\/span>\n    <span class=\"n\">LPARAM<\/span> <span class=\"n\">lParam<\/span><span class=\"p\">;<\/span>    <span class=\"c1\">\/\/ Additional info (context-specific)<\/span>\n    <span class=\"n\">POINT<\/span>  <span class=\"n\">pt<\/span><span class=\"p\">;<\/span>        <span class=\"c1\">\/\/ Mouse position<\/span>\n    <span class=\"n\">DWORD<\/span>  <span class=\"n\">lPrivate<\/span><span class=\"p\">;<\/span>  <span class=\"c1\">\/\/ Timestamp when the message was generated<\/span>\n<span class=\"p\">}<\/span> <span class=\"n\">MSG<\/span><span class=\"p\">;<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>For details, refer to the <a href=\"https:\/\/learn.microsoft.com\/en-us\/windows\/win32\/api\/winuser\/ns-winuser-msg\">official documentation<\/a>.<\/p>\n\n<h3 id=\"workflow-of-the-message-queue\">Workflow of the Message Queue<\/h3>\n\n<p>The working process of the message queue can be broken down into the following stages:<\/p>\n\n<ul>\n  <li>\n    <p><strong>Queue creation<\/strong>: Each GUI thread (i.e., a thread that creates windows) is assigned its own message queue. This queue is initialized when the thread calls GUI-related functions like <code class=\"language-plaintext highlighter-rouge\">GetMessage<\/code> for the first time. Note that message queues are <strong>thread-local<\/strong>; messages do not automatically propagate between threads.<\/p>\n  <\/li>\n  <li>\n    <p><strong>Message enqueueing<\/strong>: The Windows kernel captures input messages (keyboard, mouse) and inserts them into the appropriate thread\u2019s message queue. Application-defined messages can be enqueued asynchronously via <code class=\"language-plaintext highlighter-rouge\">PostMessage<\/code>. Synchronous messages, such as those sent by <code class=\"language-plaintext highlighter-rouge\">SendMessage<\/code>, bypass the queue and directly invoke the target window\u2019s procedure.<\/p>\n  <\/li>\n  <li>\n    <p><strong>Message retrieval and dispatch<\/strong>: A typical message loop continuously extracts messages, optionally performs pre-processing (e.g., converting virtual key presses to character messages), and dispatches them to the appropriate window procedure:<\/p>\n  <\/li>\n<\/ul>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">MSG<\/span> <span class=\"n\">msg<\/span><span class=\"p\">;<\/span>\n<span class=\"k\">while<\/span> <span class=\"p\">(<\/span><span class=\"n\">GetMessage<\/span><span class=\"p\">(<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">msg<\/span><span class=\"p\">,<\/span> <span class=\"nb\">NULL<\/span><span class=\"p\">,<\/span> <span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"mi\">0<\/span><span class=\"p\">))<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">TranslateMessage<\/span><span class=\"p\">(<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">msg<\/span><span class=\"p\">);<\/span>    <span class=\"c1\">\/\/ Translate virtual key to character message<\/span>\n    <span class=\"n\">DispatchMessage<\/span><span class=\"p\">(<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">msg<\/span><span class=\"p\">);<\/span>     <span class=\"c1\">\/\/ Send message to window procedure<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>Every window has a <strong>window procedure<\/strong> function responsible for handling its messages. Unhandled messages should be forwarded to the system-defined <code class=\"language-plaintext highlighter-rouge\">DefWindowProc<\/code>, which performs default processing. A typical example:<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">LRESULT<\/span> <span class=\"n\">CALLBACK<\/span> <span class=\"nf\">WndProc<\/span><span class=\"p\">(<\/span><span class=\"n\">HWND<\/span> <span class=\"n\">hwnd<\/span><span class=\"p\">,<\/span> <span class=\"n\">UINT<\/span> <span class=\"n\">msg<\/span><span class=\"p\">,<\/span> <span class=\"n\">WPARAM<\/span> <span class=\"n\">wParam<\/span><span class=\"p\">,<\/span> <span class=\"n\">LPARAM<\/span> <span class=\"n\">lParam<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">switch<\/span> <span class=\"p\">(<\/span><span class=\"n\">msg<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">case<\/span> <span class=\"n\">WM_PAINT<\/span><span class=\"p\">:<\/span>\n            <span class=\"c1\">\/\/ Handle painting<\/span>\n            <span class=\"k\">break<\/span><span class=\"p\">;<\/span>\n        <span class=\"k\">case<\/span> <span class=\"n\">WM_KEYDOWN<\/span><span class=\"p\">:<\/span>\n            <span class=\"c1\">\/\/ Handle key press<\/span>\n            <span class=\"k\">break<\/span><span class=\"p\">;<\/span>\n        <span class=\"c1\">\/\/ Additional message handling<\/span>\n        <span class=\"nl\">default:<\/span>\n            <span class=\"k\">return<\/span> <span class=\"n\">DefWindowProc<\/span><span class=\"p\">(<\/span><span class=\"n\">hwnd<\/span><span class=\"p\">,<\/span> <span class=\"n\">msg<\/span><span class=\"p\">,<\/span> <span class=\"n\">wParam<\/span><span class=\"p\">,<\/span> <span class=\"n\">lParam<\/span><span class=\"p\">);<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">return<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>This message dispatch architecture is fundamental to all Windows GUI applications and serves as the foundation upon which more advanced features\u2014such as custom event routing and shortcut management\u2014are built.<\/p>\n\n<h3 id=\"relationship-between-message-queue-and-keyboard-shortcuts\">Relationship Between Message Queue and Keyboard Shortcuts<\/h3>\n\n<p>When a user presses a keyboard shortcut, the Windows system generates a corresponding message\u2014typically <code class=\"language-plaintext highlighter-rouge\">WM_KEYDOWN<\/code>\u2014and places it into the message queue associated with the relevant thread (as determined by the <strong>active<\/strong> and <strong>focus<\/strong> windows). If an application has registered a <strong>global shortcut<\/strong> using <code class=\"language-plaintext highlighter-rouge\">RegisterHotKey<\/code>, the system directly sends a <code class=\"language-plaintext highlighter-rouge\">WM_HOTKEY<\/code> message to the main window procedure. In practical development, the following types of messages are most relevant to shortcut key handling:<\/p>\n\n<table>\n  <thead>\n    <tr>\n      <th>Message Type<\/th>\n      <th>Trigger Condition<\/th>\n      <th>Primary Handler<\/th>\n    <\/tr>\n  <\/thead>\n  <tbody>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">WM_KEYDOWN<\/code><\/td>\n      <td>Standard key press (excluding system keys)<\/td>\n      <td>Focused control \u2192 Main window<\/td>\n    <\/tr>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">WM_SYSKEYDOWN<\/code><\/td>\n      <td>System key press (e.g., Alt, F10)<\/td>\n      <td>Main window<\/td>\n    <\/tr>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">WM_SYSCHAR<\/code><\/td>\n      <td>Alt combined with character key<\/td>\n      <td>Focused control \u2192 Main window<\/td>\n    <\/tr>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">WM_CHAR<\/code><\/td>\n      <td>Character input within an editable control<\/td>\n      <td>Focused control \u2192 Main window<\/td>\n    <\/tr>\n    <tr>\n      <td><code class=\"language-plaintext highlighter-rouge\">WM_HOTKEY<\/code><\/td>\n      <td>Registered system-wide hotkey activation<\/td>\n      <td>Main window<\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n\n<p>If a focused control does not handle the received keyboard message, it is <strong>propagated upward<\/strong> to its parent widget or main window. If the main window also fails to process it, the message is passed to the system for default handling.<\/p>\n\n<p>An additional technical detail: in the case of <code class=\"language-plaintext highlighter-rouge\">WM_SYSCHAR<\/code>, the <code class=\"language-plaintext highlighter-rouge\">wParam<\/code> field within the message distinguishes between uppercase and lowercase characters\u2014this can be important for handling case-sensitive input.<\/p>\n\n<hr \/>\n\n<h1 id=\"comparing-native-win32-and-qt-implementations\">Comparing Native Win32 and Qt Implementations<\/h1>\n\n<h2 id=\"win32-api\">Win32 API<\/h2>\n\n<p>On Windows, the implementation of the shortcut key mechanism relies on the Win32 API. It involves direct use of the Windows message queue, with the message loop manually implemented by the developer (e.g., using <code class=\"language-plaintext highlighter-rouge\">GetMessage<\/code>, <code class=\"language-plaintext highlighter-rouge\">TranslateMessage<\/code>, <code class=\"language-plaintext highlighter-rouge\">DispatchMessage<\/code>, etc.). To retrieve the current window state, APIs such as <code class=\"language-plaintext highlighter-rouge\">GetForegroundWindow<\/code>, <code class=\"language-plaintext highlighter-rouge\">GetActiveWindow<\/code>, and <code class=\"language-plaintext highlighter-rouge\">GetFocus<\/code> are used. In typical development scenarios:<\/p>\n\n<ul>\n  <li><strong>Global shortcuts<\/strong> are registered via <code class=\"language-plaintext highlighter-rouge\">RegisterHotKey<\/code>.<\/li>\n  <li><strong>Application-level shortcuts<\/strong> are handled through <code class=\"language-plaintext highlighter-rouge\">TranslateAccelerator<\/code> within the message loop.<\/li>\n  <li><strong>Widget-level shortcuts<\/strong> are managed by processing messages like <code class=\"language-plaintext highlighter-rouge\">WM_KEYDOWN<\/code> in the window procedure.<\/li>\n<\/ul>\n\n<p>From a development perspective, programmers must manually manage both the message loop and the window procedure, while also taking care of many low-level details. This can be cumbersome, especially for teams working on cross-platform applications.<\/p>\n\n<h2 id=\"qt-framework\">Qt Framework<\/h2>\n\n<p>Although native Windows development offers greater flexibility and fine-grained control, it also introduces significant complexity\u2014making it more suitable for scenarios requiring low-level system access. In contrast, Qt simplifies event and shortcut handling by abstracting the Windows message mechanism, greatly enhancing cross-platform capability. It is well-suited for rapid development of cross-platform GUI applications, where managing shortcuts and window states becomes more convenient and intuitive.<\/p>\n\n<h3 id=\"qt-message-mechanism-abstraction\">Qt Message Mechanism Abstraction<\/h3>\n\n<p>Qt encapsulates the Windows messaging system to offer a cross-platform event system. The event loop in Qt is managed by either <code class=\"language-plaintext highlighter-rouge\">QCoreApplication<\/code> or <code class=\"language-plaintext highlighter-rouge\">QApplication<\/code>, meaning developers typically do not need to interact with the native Windows message queue directly. Qt\u2019s event system includes <code class=\"language-plaintext highlighter-rouge\">QEvent<\/code>, which abstracts various system-level messages such as keyboard events (<code class=\"language-plaintext highlighter-rouge\">QKeyEvent<\/code>), mouse events (<code class=\"language-plaintext highlighter-rouge\">QMouseEvent<\/code>), and window events. Qt also supports event filters that allow interception and customized handling of events. Moreover, the signal-slot mechanism facilitates communication between objects, partially replacing traditional message passing.<\/p>\n\n<p>The underlying message queue details are hidden from developers, who only need to focus on implementing the event handler functions. Internally, Qt translates native Windows messages into Qt events, so developers only work with these abstracted events.<\/p>\n\n<h3 id=\"window-state-management-in-qt\">Window State Management in Qt<\/h3>\n\n<p>Qt provides a unified and cross-platform API for managing window states. Internally, Qt automatically maps and synchronizes window state representations across platforms such as Windows, Linux, and macOS. This abstraction allows the same codebase to operate seamlessly across multiple platforms. Key interfaces include:<\/p>\n\n<ul>\n  <li><code class=\"language-plaintext highlighter-rouge\">QWidget::isActiveWindow()<\/code>: Determines whether a window is currently active.<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">QApplication::activeWindow()<\/code>: Returns the currently active window.<\/li>\n  <li><code class=\"language-plaintext highlighter-rouge\">QWidget::hasFocus()<\/code>: Checks if a widget currently holds input focus.<\/li>\n<\/ul>\n\n<h3 id=\"qt-shortcut-key-handling\">Qt Shortcut Key Handling<\/h3>\n\n<p>For application-level shortcuts, Qt offers the <code class=\"language-plaintext highlighter-rouge\">QShortcut<\/code> class, allowing developers to bind specific keys to widgets or windows with ease. Qt automatically manages shortcut event filtering and triggering. For widget-level key handling, one may override <code class=\"language-plaintext highlighter-rouge\">keyPressEvent<\/code> and <code class=\"language-plaintext highlighter-rouge\">keyReleaseEvent<\/code>, or use <code class=\"language-plaintext highlighter-rouge\">QAction<\/code> to bind shortcut keys to widgets or menus.<\/p>\n\n<p>However, Qt does not natively support global shortcuts (i.e., system-wide shortcuts across applications). Implementing such functionality requires direct use of platform-specific APIs (e.g., <code class=\"language-plaintext highlighter-rouge\">RegisterHotKey<\/code> on Windows) or integration with third-party libraries.<\/p>\n\n<hr \/>\n\n<h1 id=\"platform-specific-considerations-in-qt\">Platform-Specific Considerations in Qt<\/h1>\n\n<p>In addition to the previously mentioned point that Qt does not natively support global shortcuts and requires platform-specific API calls, platform discrepancies are also reflected in the behavior and default handling of shortcuts. For instance, modifier keys differ across platforms: on Windows\/Linux, they include &lt;Ctrl&gt;, &lt;Alt&gt;, and &lt;Shift&gt;, whereas on macOS, they are &lt;Command&gt;, &lt;Option&gt;, and &lt;Control&gt;. Qt provides built-in adaptations for some common key combinations\u2014for example, <code class=\"language-plaintext highlighter-rouge\">QKeySequence::Copy<\/code> maps to &lt;Ctrl+C&gt; on Windows and &lt;Command+C&gt; on macOS.<\/p>\n\n<p>When dealing with shortcuts that involve modifier keys in Qt, there are generally two approaches to handle platform differences: compile-time detection and runtime detection. <strong>Compile-time detection<\/strong> involves using conditional compilation macros to distinguish between platforms. For example:<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"cp\">#ifdef Q_OS_MAC\n<\/span>    <span class=\"n\">QShortcut<\/span> <span class=\"o\">*<\/span><span class=\"n\">shortcut<\/span> <span class=\"o\">=<\/span> <span class=\"k\">new<\/span> <span class=\"nf\">QShortcut<\/span><span class=\"p\">(<\/span><span class=\"n\">QKeySequence<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Meta+F\"<\/span><span class=\"p\">),<\/span> <span class=\"k\">this<\/span><span class=\"p\">);<\/span>\n<span class=\"cp\">#else\n<\/span>    <span class=\"n\">QShortcut<\/span> <span class=\"o\">*<\/span><span class=\"n\">shortcut<\/span> <span class=\"o\">=<\/span> <span class=\"k\">new<\/span> <span class=\"nf\">QShortcut<\/span><span class=\"p\">(<\/span><span class=\"n\">QKeySequence<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Ctrl+F\"<\/span><span class=\"p\">),<\/span> <span class=\"k\">this<\/span><span class=\"p\">);<\/span>\n<span class=\"cp\">#endif\n<\/span><\/code><\/pre><\/div><\/div>\n\n<p><strong>Runtime detection<\/strong> involves using <code class=\"language-plaintext highlighter-rouge\">QSysInfo::productType()<\/code> to determine the platform at runtime and dynamically set the key combination. For example:<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">QKeySequence<\/span> <span class=\"n\">shortcutKey<\/span><span class=\"p\">;<\/span>\n<span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">QSysInfo<\/span><span class=\"o\">::<\/span><span class=\"n\">productType<\/span><span class=\"p\">()<\/span> <span class=\"o\">==<\/span> <span class=\"s\">\"osx\"<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">shortcutKey<\/span> <span class=\"o\">=<\/span> <span class=\"n\">QKeySequence<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Meta+F\"<\/span><span class=\"p\">);<\/span>\n<span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">shortcutKey<\/span> <span class=\"o\">=<\/span> <span class=\"n\">QKeySequence<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Ctrl+F\"<\/span><span class=\"p\">);<\/span>\n<span class=\"p\">}<\/span>\n<span class=\"n\">QShortcut<\/span> <span class=\"o\">*<\/span><span class=\"n\">shortcut<\/span> <span class=\"o\">=<\/span> <span class=\"k\">new<\/span> <span class=\"n\">QShortcut<\/span><span class=\"p\">(<\/span><span class=\"n\">shortcutKey<\/span><span class=\"p\">,<\/span> <span class=\"k\">this<\/span><span class=\"p\">);<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>Of course, for common shortcut combinations, you can directly use the predefined key sequences provided by Qt.<\/p>\n\n<hr \/>\n\n<p>Related Posts \/ Websites \ud83d\udc47<\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/learn.microsoft.com\/en-us\/windows\/win32\/api\/winuser\/ns-winuser-msg\">Microsoft - Wind32 Apps MSG<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/doc.qt.io\/qt-6\/qtsystemdetection.html#macros\">Qt - System Detection Macros<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/doc.qt.io\/qt-6\/qsysinfo.html#productType\">Qt - QSysInfo<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/doc.qt.io\/qt-6\/qkeysequence.html#standard-shortcuts\">Qt - QKeySequence<\/a><\/p>\n","pubDate":"Fri, 06 Jun 2025 00:00:00 +0000","link":"https:\/\/huruilizhen.github.io\/Technical-Classification-and-Implementation-of-Hotkeys-in-Windows-Desktop-Applications","guid":"https:\/\/huruilizhen.github.io\/Technical-Classification-and-Implementation-of-Hotkeys-in-Windows-Desktop-Applications","category":["windows","desktop","hotkeys","qt"]},{"title":"Forming Coalition","description":"<p>Coalitional games are a type of game theoretical model that study scenarios where agents can reap mutual benefits from cooperation. In such games, a coalition is a group of agents that work together and obtain a payoff for their collective effort. The agents are considered \u201ccooperative\u201d since cooperation yields them more payoffs than non-cooperation. This post discusses on coalition in Lecture 11 in the <code class=\"language-plaintext highlighter-rouge\">SC4003<\/code> course in NTU. See details below!<\/p>\n\n<!--more-->\n\n<h2 id=\"table-of-contents\">Table of Contents<\/h2>\n<ul>\n  <li><a href=\"#coalitional-game-is-a-cooperative-game\">Coalitional Game is a Cooperative Game<\/a>\n    <ul>\n      <li><a href=\"#coalition-structure-generation\">Coalition Structure Generation<\/a><\/li>\n      <li><a href=\"#coalition-values---benefit-of-cooperation\">Coalition Values - Benefit of Cooperation<\/a><\/li>\n      <li><a href=\"#dividing-coalition-values\">Dividing Coalition Values<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#some-solutions\">Some solutions<\/a>\n    <ul>\n      <li><a href=\"#the-core\">The Core<\/a><\/li>\n      <li><a href=\"#the-shapley-value\">The Shapley Value<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#applications\">Applications<\/a><\/li>\n<\/ul>\n\n<hr \/>\n\n<h1 id=\"coalitional-game-is-a-cooperative-game\">Coalitional Game is a Cooperative Game<\/h1>\n\n<p>Key Issues in Coalitional Games (Sandholm 1999):<\/p>\n<ul>\n  <li><strong>Coalition Structure Generation<\/strong>: The process of identifying and forming optimal coalitions among agents to maximize collective benefits.<\/li>\n  <li><strong>Coalition Values<\/strong>: Assessment of the benefits derived from cooperation within a coalition, representing the total value created by the coalition.<\/li>\n  <li><strong>Division of Coalition Values<\/strong>: Determination of a fair and equitable distribution of the total coalition value among its members based on predefined rules or principles.<\/li>\n<\/ul>\n\n<h2 id=\"coalition-structure-generation\">Coalition Structure Generation<\/h2>\n\n<p>When agents join coalitions, the overall state is a coalition structure. And there are two types of coalition structure:<\/p>\n\n<ul>\n  <li>Allowing overlapping<\/li>\n  <li>No overlapping, also called partitions.Partition agents into disjoint coalitions. The overall partition is a coalition structure. In this post we will mainly focus on non-overlapping coalitions.<\/li>\n<\/ul>\n\n<h2 id=\"coalition-values---benefit-of-cooperation\">Coalition Values - Benefit of Cooperation<\/h2>\n\n<p>Given a set of players or agents:<\/p>\n\n\\[Ag = \\{1, 2, \\ldots, n\\}\\]\n\n<p>Coalition values have a form of a characteristic function:<\/p>\n\n\\[v: 2^{Ag} \\rightarrow \\mathbb{R}\\]\n\n<p>There are two types of coalition values:<\/p>\n<ul>\n  <li>Transferable coalition value, a.k.a  transferable utility (TU), which we will focus on.<\/li>\n  <li>Non-transferable coalition value, which is a bit more complex.<\/li>\n<\/ul>\n\n<h2 id=\"dividing-coalition-values\">Dividing Coalition Values<\/h2>\n\n<p>Having gained the coalition values, how to allocate the values to each agent is a key issue in coalition games. Define the coalition $S$ a subset of agent set:<\/p>\n\n\\[S \\subseteq Ag\\]\n\n<p>The allocation value to each agent $i \\in S$ is a vector of size $|S|$:<\/p>\n\n\\[\\mathbf{x} = \\langle x_1, x_2, \\ldots, x_{|S|} \\rangle \\in \\mathbb{R}^{|S|}\\]\n\n<p>A feasible distribution should satisfy:<\/p>\n\n\\[\\sum_{i \\in S} x_i \\leq v(S)\\]\n\n<p>Another issue is how to make the coalition stable. Will any agents defect from the coalition? If coalition try to give individuals a bad payoff, individuals can always walk away.<\/p>\n\n<hr \/>\n\n<h1 id=\"some-solutions\">Some solutions<\/h1>\n\n<h2 id=\"the-core\">The Core<\/h2>\n\n<p>The core of a coalitional game is a set of <strong>feasible distributions<\/strong> of payoffs to members of a coalition that <strong>no sub-coalition can reasonably object<\/strong> to the grand coalition. Define the grand coalition as subset $S^\\prime$ of all agents:<\/p>\n\n\\[S^\\prime \\subseteq Ag\\]\n\n<p>A distribution $\\mathbf{x}$ is called <strong>core<\/strong> if it is satisfies <strong>efficiency<\/strong>, the total distribution cannot exceed the total value that the grand coalition can bring:<\/p>\n\n\\[\\sum_{i \\in S^\\prime} x_i = v(S^\\prime)\\]\n\n<p>and <strong>coalitional rationalit<\/strong>, which means for any subset $S_{sub}$ of $S^\\prime$, the total benefit allocated to them should not be less than what they can obtain by working alone.<\/p>\n\n\\[\\sum_{i \\in S_{sub}} x_i \\geq v(S_{sub})\\]\n\n<p>It the a subset $S_{sub}$ of $S^\\prime$ satisfies following condition, we say it is a <strong>objection<\/strong>:<\/p>\n\n\\[x^{sub}_i &gt; x^\\prime_i \\quad \\forall i \\in S_{sub}\\]\n\n<p>which means we have a sub-coalition that can object to the grand coalition because it gets more benefit from working alone. If the grand coalition has no objection, then the distribution is called <strong>core<\/strong>.<\/p>\n\n<p>We can tell that the core is not always unique, suppose:<\/p>\n\n\\[Ag = \\{1, 2\\}, \\quad v(\\{1\\}) = 5, \\quad v(\\{2\\}) = 5, \\quad v(\\{1, 2\\}) = 20\\]\n\n<p>Then, by the definition of core, we actually have many cores. And one of them maybe:<\/p>\n\n\\[\\langle 5, 15 \\rangle\\]\n\n<p>But it seems not fair to agent $1$. The following one is much more fair:<\/p>\n\n\\[\\langle 10, 10 \\rangle\\]\n\n<p>Hence, we need to find a way to fairly distribute the coalition values.<\/p>\n\n<h2 id=\"the-shapley-value\">The Shapley Value<\/h2>\n\n<p>The Shapley value is a best known attempt to define how to divide coalition value fairly. The Shapley value of an agent i is the average\namount of agent $i$\u2019s marginal contribution:<\/p>\n\n\\[\\phi_i(v) = \\sum_{S_{sub} \\subseteq S^\\prime - \\{i\\}} \\frac{\\|S_{sub}\\|!(\\|S^\\prime - \\{i\\}\\| - \\|S_{sub}\\| - 1)!}{\\|S^\\prime - \\{i\\}\\|!} (v(S_{sub} \\cup \\{i\\}) - v(S_{sub}))\\]\n\n<p>where $n!$ is the factorial of $n$<\/p>\n\n<p>Imagine the coalition being formed one agent at a time, with each agent demanding their contribution:<\/p>\n\n\\[v(S_{sub} \\cup \\{i\\}) - v(S_{sub})\\]\n\n<p>as a fair compensation, and then averaging over the possible different permutations in which the coalition can be formed.<\/p>\n\n<p>The Shapley value has four properties:<\/p>\n<ul>\n  <li>\n    <p><strong>Efficiency<\/strong>: the total payoff is distributed<\/p>\n\n\\[\\sum_{i \\in S^\\prime} \\phi_i(v) = v(S^\\prime)\\]\n  <\/li>\n  <li>\n    <p><strong>Symmetry<\/strong>: if agents $i$ and $j$ are equivalent in a sense that<\/p>\n\n\\[S_{sub} \\cap \\{i, j\\} = \\emptyset \\wedge v(S_{sub} \\cup \\{i\\}) = v(S_{sub} \\cup \\{j\\}) \\implies\\phi_i(v) = \\phi_j(v)\\]\n  <\/li>\n  <li>\n    <p><strong>Additivity<\/strong>:<\/p>\n\n\\[\\phi_i(v) + \\phi_i(w) = \\phi(v+w)\\]\n  <\/li>\n  <li>\n    <p><strong>Dummy agent (null agent)<\/strong>:<\/p>\n\n\\[v(S^\\prime \\cup ag_0) = v(S^\\prime)\\]\n  <\/li>\n<\/ul>\n\n<p>The Shapley value <strong>may or may not be in the Core<\/strong>, as it is a <strong>fairness-oriented solution concept<\/strong> that does not necessarily consider the stability of the coalition. The Core, on the other hand, is a <strong>stability-oriented solution concept<\/strong> that ensures no subset of agents can do better by forming a coalition outside of the grand coalition. Therefore, there is no direct relationship between the Shapley value and the Core.<\/p>\n\n<p>According to Shapley value equation, naive, obvious representation of coalitional game is <strong>exponential<\/strong> in the size of $n$. Not practical. Hence we introduce <strong>Induced subgraph<\/strong> (succinct but incomplete) and <strong>Marginal contribution nets<\/strong> (not succinct but complete).<\/p>\n\n<p>For <strong>Induced subgraph<\/strong>, we represents $v$ as an undirected graph on $S^\\prime$, with integer weights $w_{i,j}$ between ndoes $i, j \\in S^\\prime$:<\/p>\n\n\\[w_{i,j} \\in \\mathbb{Z} \\quad i, j \\in S^\\prime\\]\n\n<p>Value of a coalition $S_{sub}$ is then:<\/p>\n\n\\[v(S_{sub}) = \\sum_{(i, j) \\in S_{sub}} w_{i,j}\\]\n\n<p>The Shapley value of agent $i$ is:<\/p>\n\n\\[\\phi_i(v) = \\frac{1}{2} \\sum w_{i, j}\\]\n\n<p>If there is a self-loop, the self-loop weight is added to the agent\u2019s Shapley value. Lets take an example:<\/p>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/induced-graph.png\" alt=\"Induced Graph\" \/>\n  \n  <figcaption class=\"caption-text\">induced graph<\/figcaption>\n  \n<\/figure>\n\n<p>For <strong>Marginal contribution nets<\/strong>, the basic idea is to use a set of rules to describe the payoffs of coalitions:<\/p>\n\n\\[\\text{pattern} \\rightarrow \\text{value}\\]\n\n<p>The value of a coalition $S_{sub}$ is the sum of all the payoffs of rules satisfied by that coalition.<\/p>\n\n\\[v(S_{sub}) = \\sum_{r \\in \\{Rules\\}} \\text{value}_{r}\\]\n\n<p>The Shapley value of an agent in an MCN is equal to the sum of the Shapley values of the agent over each rule:<\/p>\n\n\\[\\phi_i(v) = \\sum_{r \\in \\{Rules\\}} \\phi_i^r(v), \\quad \\phi_i^r(v) = \\frac{value_r}{\\|r\\|}\\]\n\n<p>An Example, let\u2019s have a set of rules:<\/p>\n\n\\[\\{a,b\\} \\rightarrow 5, \\quad \\{b\\} \\rightarrow 2\\]\n\n<p>Then we get that:<\/p>\n\n\\[v(\\{a,b\\}) = 5, \\quad v(\\{b\\}) = 2, \\quad v(\\{a\\}) = 0\\]\n\n<p>The Shapley value of agent $a$ is:<\/p>\n\n\\[\\phi_a(v) = \\frac{5}{2} + \\frac{0}{1} = 2.5\\]\n\n<p>The Shapley value of agent $b$ is:<\/p>\n\n\\[\\phi_b(v) = \\frac{5}{2} + \\frac{2}{1} = 4.5\\]\n\n<hr \/>\n\n<h1 id=\"applications\">Applications<\/h1>\n\n<p><strong>Multi-Agent Systems Coordination<\/strong> and <strong>Explainable Artificial Intelligence<\/strong> are two areas where Shapley values are being applied.<\/p>\n\n<p>In multi-agent systems coordination, Shapley values are used to fairly allocate tasks and resources among self-interested agents. This is particularly useful in <strong>team formation<\/strong> and <strong>collaborative computing<\/strong>, where the goal is to maximize the overall performance of the team.<\/p>\n\n<p>In explainable AI, Shapley values are used to assign importance scores to features in machine learning models. This is useful in high-stakes applications where the decisions made by the model need to be <strong>interpretable<\/strong> and <strong>transparent<\/strong>. The Shapley value approach was introduced by SHAP in 2017 and has since become a widely used method for explaining complex models. It has been used to interpret even very large and highly complex models.<\/p>\n\n<hr \/>\n\n<p>Related Posts \/ Websites \ud83d\udc47<\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Intelligent-Agent-Prolegomenon\">Ray - NTU SC4003 Lecture 3 Note: Intelligent Agent Prolegomenon<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Agent-Decision-Making\">Ray - NTU SC4003 Lecture 4 Note: Agent Decision Making<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Agent-Architecture\">Ray - NTU SC4003 Lecture 5 Note: Agent Architecture<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Working-Together-Benevolent-Cooperative-Agents\">Ray - NTU SC4003 Lecture 7 Note: Working Together Benevolent\/Cooperative Agents<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Self-Interested-Agents-Game-Theory-Foundation\">Ray - NTU SC4003 Lecture 8 Note: Game Theory Foundations<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"Allocating-Scarce-Resources-Auction#vickrey-auctions\">Ray - NTU SC4003 Lecture 9 Note: Allocating Scarce Resources: Auction<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Making-Group-Decisions-Voting\">Ray - NTU SC4003 Lecture 10 Note: Voting Mechanism<\/a><\/p>\n","pubDate":"Fri, 18 Apr 2025 00:00:00 +0000","link":"https:\/\/huruilizhen.github.io\/Forming-Coalition","guid":"https:\/\/huruilizhen.github.io\/Forming-Coalition","category":"intelligent-agent"},{"title":"Making Group Decisions: Voting","description":"<p>In the previous post, we examined how agents make decisions in two-agent games, focusing on the <strong>Nash Equilibrium<\/strong>. In this post, we will explore how a group of agents make decisions, delving into the realm of social choice theory. A classic example of social choice theory is <strong>voting<\/strong>, where the challenge is to combine individual preferences to derive a social outcome. This post discusses voting mechanism in group decision making in Lecture 10 of the <code class=\"language-plaintext highlighter-rouge\">SC4003<\/code> course in NTU. Let\u2019s begin!<\/p>\n\n<!--more-->\n\n<h2 id=\"table-of-contents\">Table of Contents<\/h2>\n\n<ul>\n  <li><a href=\"#notations-and-definitions\">Notations and Definitions<\/a>\n    <ul>\n      <li><a href=\"#components-of-a-social-choice-model\">Components of a Social Choice Model<\/a><\/li>\n      <li><a href=\"#preferences\">Preferences<\/a><\/li>\n      <li><a href=\"#social-welfare-functions\">Social Welfare Functions<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#voting-procedures-plurality\">Voting Procedures: Plurality<\/a>\n    <ul>\n      <li><a href=\"#anomalies-with-plurality\">Anomalies with Plurality<\/a><\/li>\n      <li><a href=\"#strategic-manipulation-by-tactical-voting\">Strategic Manipulation by Tactical Voting<\/a><\/li>\n      <li><a href=\"#condorcets-paradox\">Condorcet\u2019s Paradox<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#sequential-majority-elections\">Sequential Majority Elections<\/a>\n    <ul>\n      <li><a href=\"#anomalies-with-sequential-pairwise-elections\">Anomalies with Sequential Pairwise Elections<\/a><\/li>\n      <li><a href=\"#majority-graphs\">Majority Graphs<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#voting-procedures-borda-count\">Voting Procedures: Borda Count<\/a>\n    <ul>\n      <li><a href=\"#desirable-properties-of-voting-procedures\">Desirable Properties of Voting Procedures<\/a><\/li>\n      <li><a href=\"#arrows-theorem\">Arrow\u2019s Theorem<\/a><\/li>\n      <li><a href=\"#the-gibbard-satterthwaite-theorem\">The Gibbard-Satterthwaite Theorem<\/a><\/li>\n    <\/ul>\n  <\/li>\n<\/ul>\n\n<hr \/>\n\n<h1 id=\"notations-and-definitions\">Notations and Definitions<\/h1>\n\n<p>Notation system make life easier when we are talking about something like voting. Let\u2019s have a short definition of game theory notations, which will make the rest of this post easier to understand.<\/p>\n\n<h2 id=\"components-of-a-social-choice-model\">Components of a Social Choice Model<\/h2>\n\n<p>The set of voters (a.k.a agents in the context of the course) is defined as:<\/p>\n\n\\[Ag = \\{1, 2, \\ldots, n\\}\\]\n\n<p>Voters make group decisions with respect to a set of \u201coutcomes\u201d or \u201ccandidates\u201d:<\/p>\n\n\\[\\Omega = \\{\\omega_1, \\omega_2, ... \\omega_3\\}\\]\n\n<p>If $| \\Omega | = 2$, say we have a pairwise election.<\/p>\n\n<h2 id=\"preferences\">Preferences<\/h2>\n\n<p>Each voter has preferences over $\\Omega$. It is an <strong>ordering<\/strong> over the set of possible outcomes $\\Omega$. Take following wines as the example:<\/p>\n\n\\[\\Omega = \\{ \\text{gin}, \\text{rum}, \\text{brandy}, \\text{whisky}\\}\\]\n\n<p>The preference of agent $ray$ is denoted as:<\/p>\n\n\\[\\bar{\\omega}_{ray} = \\langle \\text{whisky}, \\text{rum}, \\text{gin}, \\text{brandy} \\rangle\\]\n\n<p>which means that agent $ray$ prefers whisky to rum, rum to gin, gin to brandy, and brandy to whisky. These can also be written as:<\/p>\n\n\\[\\text{whisky} \\succ_{ray} \\text{rum} \\succ_{ray} \\text{gin} \\succ_{ray} \\text{brandy}\\]\n\n<p>The fundamental problem of <strong>social choice theory<\/strong> is to determine <strong>how to aggregate individual preference orders<\/strong> of voters into a collective decision that accurately represents the preferences of the group. This involves understanding how to merge diverse individual preferences into a coherent group preference. There are two primary approaches to preference aggregation: <strong>social welfare functions<\/strong>, which focus on ranking social states based on individual preferences, and <strong>social choice functions<\/strong>, which aim to select a single outcome from a set of alternatives.<\/p>\n\n<h2 id=\"social-welfare-functions\">Social Welfare Functions<\/h2>\n\n<p>The set of preference orderings over $\\Omega$ is denoted as:<\/p>\n\n\\[\\pi(\\Omega)\\]\n\n<p>A social welfare function $f$ takes the <strong>voter preferences order<\/strong> and produces a <strong>social preference order<\/strong>, e.g. beauty contest:<\/p>\n\n\\[f: \\pi(\\Omega) \\times \\pi(\\Omega) \\times \\ldots \\times \\pi(\\Omega) \\rightarrow \\pi(\\Omega)\\]\n\n<p>To denote the <strong>outcome comparison<\/strong> of a social welfare function:<\/p>\n\n\\[\\succ^*\\]\n\n<p>Sometimes, we want to select one of the possible candidates, rather than a social order. This gives social choice functions $f$, e.g. presidential election:<\/p>\n\n\\[f: \\pi(\\Omega) \\times \\pi(\\Omega) \\times \\ldots \\times \\pi(\\Omega) \\rightarrow \\Omega\\]\n\n<hr \/>\n\n<h1 id=\"voting-procedures-plurality\">Voting Procedures: Plurality<\/h1>\n\n<p><strong>Social Choice Function<\/strong>, which selects a single outcome according to all <strong>preferences<\/strong> of the voters. Each candidate gets one point for every <strong>preference order<\/strong> that ranks them <strong>first<\/strong>. The <strong>winner<\/strong> is the one with the <strong>largest number of points<\/strong>. For example, <strong>Political elections in UK<\/strong>. If we have only two candidates, then <strong>plurality<\/strong> is a simple <strong>majority election<\/strong>.<\/p>\n\n<h2 id=\"anomalies-with-plurality\">Anomalies with Plurality<\/h2>\n\n<p>One anomaly with the plurality voting procedure is that it can produce a winner that is <strong>not supported by a majority of voters<\/strong>. For example, consider an election with $100$ voters and three candidates $\\omega_1$, $\\omega_2$, and $\\omega_3$ where the distribution of votes is as follows:<\/p>\n<ul>\n  <li>$40%$ of voters vote for $\\omega_1$<\/li>\n  <li>$30%$ of voters vote for $\\omega_2$<\/li>\n  <li>$30%$ of voters vote for $\\omega_3$<\/li>\n<\/ul>\n\n<p>In this scenario, \u03c91 would win the election with $40\\%$ of the votes, even though a clear majority ($60\\%$) of the voters prefer another candidate. This is a limitation of the plurality voting procedure in that it does not ensure that the winner is supported by a majority of the voters.<\/p>\n\n<h2 id=\"strategic-manipulation-by-tactical-voting\">Strategic Manipulation by Tactical Voting<\/h2>\n\n<p>The concept of strategic manipulation by tactical voting is a phenomenon where a voter may vote for a candidate other than their true preference in order to achieve a better outcome. This can occur when a voter believes that their true preference is not viable, and that by voting for a second choice, they can prevent a less desirable candidate from winning. This is also known as <strong>strategic voting<\/strong> or <strong>insincere voting<\/strong>. For example, if a voter\u2019s true preference is candidate $A$, but they believe that candidate $B$ is more likely to win than $A$, they may vote for $B$ in order to prevent candidate $C$ from winning, even if $B$ is not their true preference.<\/p>\n\n\\[\\text{Strategic voting: } \\text{Vote for } B \\text{ to prevent } C \\text{ from winning}\\]\n\n<p>In this scenario, the voter is engaging in strategic manipulation of the vote, as they are not voting for their true preference. This can lead to a situation where the outcome of the election does not accurately reflect the true preferences of the voters. Let\u2019s take a look at following example, suppose my preferences are:<\/p>\n\n\\[\\omega_1 \\succ \\omega_2 \\succ \\omega_3\\]\n\n<p>I believe $48.9\\%$ of voters have preferences:<\/p>\n\n\\[\\omega_2 \\succ \\omega_1 \\succ \\omega_3\\]\n\n<p>and I believe $49.1\\%$ of voters have preferences<\/p>\n\n\\[\\omega_3 \\succ \\omega_2 \\succ \\omega_1\\]\n\n<p>Then if I can represent $2\\%$ of voters, I may do better voting for $\\omega_2$ , even though this is not my true preference profile.<\/p>\n\n<h2 id=\"condorcets-paradox\">Condorcet\u2019s Paradox<\/h2>\n\n<p>Condorcet\u2019s paradox arises in voting scenarios where a <strong>majority preference cycle<\/strong> exists. Given a set of agents $Ag = {1, 2, 3}$ and outcomes $\\Omega = {\\omega_1, \\omega_2, \\omega_3}$, consider the preferences:<\/p>\n\n\\[\\begin{align*}\n  \\omega_1 &amp; \\succ_1 \\omega_2 \\succ_1 \\omega_3 \\\\\n  \\omega_3 &amp; \\succ_2 \\omega_1 \\succ_2 \\omega_2 \\\\\n  \\omega_2 &amp; \\succ_3 \\omega_3 \\succ_3 \\omega_1\n\\end{align*}\\]\n\n<p>In this setup, for any candidate chosen, there exists another candidate preferred by a majority, leading to no clear winner. This exemplifies Condorcet\u2019s paradox, where no option satisfies the majority.<\/p>\n\n<hr \/>\n\n<h1 id=\"sequential-majority-elections\">Sequential Majority Elections<\/h1>\n\n<p>A variant of plurality, in which players play in a series of rounds: either a linear sequence or a tree (knockout tournament). We can use an example to understand this:<\/p>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/sequential-pairwise-election.png\" alt=\"Sequential Majority Elections\" \/>\n  \n  <figcaption class=\"caption-text\">sequential majority elections<\/figcaption>\n  \n<\/figure>\n\n<h2 id=\"anomalies-with-sequential-pairwise-elections\">Anomalies with Sequential Pairwise Elections<\/h2>\n\n<p>Here, we pick an ordering of the outcomes,which determines who plays against who.For example, if the agenda is:<\/p>\n\n\\[\\langle \\omega_2, \\omega_3, \\omega_4, \\omega_1 \\rangle\\]\n\n<p>Then the first election is between $\\omega_2$ and $\\omega_3$ , and the winner goes on to an election with \\omega_4, and the winner of this election goes in an election with $\\omega_1$. Anomalies can occur in sequential pairwise elections because the winner of an election may not be supported by a majority of the voters but depend on the <strong>agenda<\/strong>. Suppose we have:<\/p>\n\n\\[\\begin{align*}\n        \\text{33 voters prefer } &amp; \\omega_1 \\succ \\omega_2 \\succ \\omega_3 \\\\\n        \\text{33 voters prefer } &amp; \\omega_3 \\succ \\omega_1 \\succ \\omega_2 \\\\\n        \\text{33 voters prefer } &amp; \\omega_2 \\succ \\omega_3 \\succ \\omega_1\n  \\end{align*}\\]\n\n<p>Then for every candidate, we can fix an agenda for that candidate to win in a sequential pairwise election! This can be demonstrated better using a <strong>majority graph<\/strong>.<\/p>\n\n<h2 id=\"majority-graphs\">Majority Graphs<\/h2>\n\n<p>This idea is easiest to illustrate by using a majority graph, which is a directed graph with:<\/p>\n<ul>\n  <li>vertices represent candidates<\/li>\n  <li>an edge $(i, j)$ if $i$ would beat $j$ is a <strong>simple majority election<\/strong>.<\/li>\n<\/ul>\n\n<p>It is a compact representation of voter preferences. Let\u2019s take a look at an example:<\/p>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/majority-graph.png\" alt=\"Majority Graph\" \/>\n  \n  <figcaption class=\"caption-text\">majority graph<\/figcaption>\n  \n<\/figure>\n\n<p>A Condorcet winner is a candidate that would beat every other candidate in a pairwise election. Like:<\/p>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/condorcet-winners.png\" alt=\"Condorcet Winner\" \/>\n  \n  <figcaption class=\"caption-text\">condorcet winner<\/figcaption>\n  \n<\/figure>\n\n<hr \/>\n\n<h1 id=\"voting-procedures-borda-count\">Voting Procedures: Borda Count<\/h1>\n\n<p>The Borda count is a voting procedure that <strong>takes into account the whole preference order of each voter<\/strong>, unlike the plurality voting procedure which only considers the top ranked candidate. In the Borda count, each candidate is <strong>assigned a score based on its position in each voter\u2019s preference order<\/strong>. The score is calculated as follows: if a candidate appears first in a preference order, its score is incremented by $k-1$, where k is the number of candidates; if it appears second, its score is incremented by $k-2$, and so on, until the last candidate in the preference order has its score incremented by $0$. After all voters have been considered, the candidate with the highest score is declared the winner. This procedure is <strong>more representative of the voters\u2019 true preferences<\/strong> than the plurality procedure, as it takes into account the strength of opinion in favour of each candidate.<\/p>\n\n<h2 id=\"desirable-properties-of-voting-procedures\">Desirable Properties of Voting Procedures<\/h2>\n\n<p>A desirable voting procedure should satisfy two fundamental properties: the <strong>Pareto property<\/strong> and <strong>Independence of Irrelevant Alternatives (IIA)<\/strong>.<\/p>\n\n<ul>\n  <li>\n    <p>The <strong>Pareto property<\/strong> states that if all voters prefer candidate $\\omega_i$ over candidate $\\omega_j$, then the social choice should also rank $\\omega_i$ higher than $\\omega_j$. This is a basic requirement that the <strong>social choice should respect the unanimous preference of the voters<\/strong>.<\/p>\n  <\/li>\n  <li>\n    <p>The <strong>IIA property<\/strong> states that the <strong>relative ranking of candidates<\/strong> $\\omega_i$ and $\\omega_j$ should only <strong>depend on the relative ranking<\/strong> of $\\omega_i$ and $\\omega_j$ in <strong>each voter\u2019s preference order<\/strong>, and should not be affected by the <strong>introduction or removal of \u201cirrelevant\u201d alternatives<\/strong>. This property ensures that the social choice is focused on the true relative preferences of the voters, and is not influenced by external factors.<\/p>\n  <\/li>\n<\/ul>\n\n<p>In summary, a desirable voting procedure should satisfy the <strong>Pareto property and the IIA property<\/strong>, which are two fundamental requirements for a fair and reasonable social choice.<\/p>\n\n<h2 id=\"arrows-theorem\">Arrow\u2019s Theorem<\/h2>\n\n<p>This theorem, proposed by economist Kenneth Arrow in 1951, reveals the fundamental impossibility of designing a fair and democratic voting system. The theorem states that:<\/p>\n\n<p>In a situation with <strong>three or more candidates<\/strong>, any voting system that satisfies the <strong>Pareto property<\/strong> and the <strong>IIA property<\/strong>, must be a <strong>dictatorship<\/strong>, meaning that the final social ranking is determined by a single individual\u2019s personal preference, without considering the opinions of others.<\/p>\n\n<p>We initially hoped to design a fair and democratic voting system that respects the preferences of all individuals and is not influenced by irrelevant alternatives.<\/p>\n\n<p>Arrow\u2019s theorem proves that such a system is technically impossible, unless we allow a single individual to determine the outcome (i.e., a dictatorship).<\/p>\n\n<h2 id=\"the-gibbard-satterthwaite-theorem\">The Gibbard-Satterthwaite Theorem<\/h2>\n\n<p>We already saw that sometimes, voters can benefit by s<strong>trategically misrepresenting their preferences<\/strong>. Are there any voting methods which are non-manipulable, in the sense that voters can never benefit from misrepresenting preferences? The answer is given by the <strong>Gibbard-Satterthwaite Theorem<\/strong>\uff1a<\/p>\n\n<blockquote>\n  <p>In votes with three or more candidates, if a voting system is deterministic, non-authoritarian, and always produces a winner, then the system can be strategically manipulated.<\/p>\n<\/blockquote>\n\n<p>which means no reasonable voting system can completely prevent voters from changing the outcome through dishonest voting. However, a news is that <strong>Computational Complexity can Help Rescue<\/strong>!<\/p>\n\n<p>Bartholdi, Tovey, and Trick showed that there are elections that are prone to manipulation in principle, but where manipulation was computationally complex.\u201dSingle Transferable Vote\u201d is a voting method that is NP-hard to manipulate!<\/p>\n\n<hr \/>\n\n<p>Related Posts \/ Websites \ud83d\udc47<\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Intelligent-Agent-Prolegomenon\">Ray - NTU SC4003 Lecture 3 Note: Intelligent Agent Prolegomenon<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Agent-Decision-Making\">Ray - NTU SC4003 Lecture 4 Note: Agent Decision Making<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Agent-Architecture\">Ray - NTU SC4003 Lecture 5 Note: Agent Architecture<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Working-Together-Benevolent-Cooperative-Agents\">Ray - NTU SC4003 Lecture 7 Note: Working Together Benevolent\/Cooperative Agents<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Self-Interested-Agents-Game-Theory-Foundation\">Ray - NTU SC4003 Lecture 8 Note: Game Theory Foundations<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"Allocating-Scarce-Resources-Auction#vickrey-auctions\">Ray - NTU SC4003 Lecture 9 Note: Allocating Scarce Resources: Auction<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Forming-Coalition\">Ray - NTU SC4003 Lecture 11 Note: Forming Coalition<\/a><\/p>\n","pubDate":"Sat, 12 Apr 2025 00:00:00 +0000","link":"https:\/\/huruilizhen.github.io\/Making-Group-Decisions-Voting","guid":"https:\/\/huruilizhen.github.io\/Making-Group-Decisions-Voting","category":"intelligent-agent"},{"title":"Allocating Scarce Resources: Auction","description":"<p>The allocation of <strong>scarce resources<\/strong>, such as physical objects or computing resources, among multiple agents is a crucial aspect of multi-agent systems. When resources are abundant or uncontested, allocation is straightforward. However, in reality, we often face scarcity and competition, which is where <strong>auction mechanisms<\/strong> come into play. Once considered rare, auctions have become a common method for resource allocation in various domains. We will cover the auction mechanism in Lecture 9 in the <code class=\"language-plaintext highlighter-rouge\">SC4003<\/code> course at NTU. Get ready!<\/p>\n\n<!--more-->\n\n<h2 id=\"table-of-contents\">Table of Contents<\/h2>\n<ul>\n  <li><a href=\"#basis-of-auction-mechanism\">Basis of Auction Mechanism<\/a>\n    <ul>\n      <li><a href=\"#limit-price\">Limit Price<\/a><\/li>\n      <li><a href=\"#market-institution\">Market Institution<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#the-zoology-of-auctions\">The Zoology of Auctions<\/a>\n    <ul>\n      <li><a href=\"#single-vs-multi-dimensional-auctions\">Single vs. Multi-Dimensional Auctions<\/a><\/li>\n      <li><a href=\"#single-vs-double-sided-auctions\">Single vs Double-Sided Auctions<\/a><\/li>\n      <li><a href=\"#open-cry-vs-sealed-bid-auctions\">Open-Cry vs. Sealed-Bid Auctions<\/a><\/li>\n      <li><a href=\"#single-vs-multi-unit-auctions\">Single vs. Multi-Unit Auctions<\/a><\/li>\n      <li><a href=\"#first-vs-k-th-price-auctions\">First vs K-th Price Auctions<\/a><\/li>\n      <li><a href=\"#single-vs-multi-item-auctions\">Single vs. Multi-Item Auctions<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#standard-auction-types\">Standard Auction Types<\/a>\n    <ul>\n      <li><a href=\"#english-auctions\">English Auctions<\/a><\/li>\n      <li><a href=\"#dutch-auctions\">Dutch Auctions<\/a><\/li>\n      <li><a href=\"#first-price-sealed-bid-auctions\">First-Price Sealed Bid Auctions<\/a><\/li>\n      <li><a href=\"#vickrey-auctions\">Vickrey Auctions<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#combinatorial-auctions\">Combinatorial Auctions<\/a>\n    <ul>\n      <li><a href=\"#notations-and-definitions\">Notations and Definitions<\/a><\/li>\n      <li><a href=\"#bidding-languages\">Bidding Languages<\/a><\/li>\n      <li><a href=\"#the-vcg-mechanism\">The VCG Mechanism<\/a><\/li>\n    <\/ul>\n  <\/li>\n<\/ul>\n\n<hr \/>\n\n<h1 id=\"basis-of-auction-mechanism\">Basis of Auction Mechanism<\/h1>\n\n<p>The auction mechanism is concerned with the allocation of <strong>goods and money between traders<\/strong>. It assumes an initial allocation, and then allows for the free exchange of goods and money between traders to alter their allocations. The <strong>goods<\/strong> being traded are assumed to be <strong>indivisible<\/strong>, while the <strong>money<\/strong> is <strong>divisible<\/strong>.<\/p>\n\n<h2 id=\"limit-price\">Limit Price<\/h2>\n\n<p>Each trader has a <strong>value<\/strong> or <strong>limit price<\/strong> that they place on the good. A buyer who exchanges more than their limit price for a good makes a loss while a seller who exchanges less than their limit price for a good makes a profit. We can tell that limit prices clearly have an effect on the behavior of traders.<\/p>\n\n<p>There are several models, embodying different assumptions about the nature of the good. Following are typical models that often found literature:<\/p>\n<ul>\n  <li><strong>Private Value<\/strong>: Good has independent value for each trader.<\/li>\n  <li><strong>Common Value<\/strong>: Good has the same value for all traders, but  estimates may be different.<\/li>\n  <li><strong>Correlated Value<\/strong>: Values are related. E.g., the more you are prepared to pay, the more I should be prepared to pay.<\/li>\n<\/ul>\n\n<h2 id=\"market-institution\">Market Institution<\/h2>\n\n<p>A market institution defines how the exchange takes place. It not only define what <strong>messages<\/strong> can be exchanged, but also defines how the final <strong>allocation<\/strong> depends on the messages. The change of allocation is so called <strong>market clearing<\/strong>. Say<\/p>\n\n<blockquote>\n  <p>An <strong>auction<\/strong> is a market institution in which messages from traders include some price information \u2014 this information may be an offer to buy at a given price, in the case of a <strong>bid<\/strong>, or an offer to sell at a given price, in the case of an <strong>ask<\/strong> \u2014 and which gives priority to higher bids and lower asks.<\/p>\n<\/blockquote>\n\n<hr \/>\n\n<h1 id=\"the-zoology-of-auctions\">The Zoology of Auctions<\/h1>\n\n<p>Auctions can be divided into a number of different categories. Being good computer scientists, we draw up a <strong>taxonomy<\/strong>. This gives us a handle on all the kinds there might be, suggests parameterization, and help us to think about implementation. This way of classification is a little bit like <strong>zoology<\/strong>, but still a good way to think about things.<\/p>\n\n<h2 id=\"single-vs-multi-dimensional-auctions\">Single vs. Multi-Dimensional Auctions<\/h2>\n\n<p>Single-dimensional auctions refer to the case where the only content of an offer are the price and quantity of some specific type of good. Whereas multi-dimensional auctions refer to the case where offers can relate to many different aspects of many different goods.<\/p>\n\n<blockquote>\n  <p>\u201cI\u2019ll bid $200 for those 2 chairs\u201d<\/p>\n<\/blockquote>\n\n<p>This is a single-dimensional since the price is only about the goods them.<\/p>\n\n<blockquote>\n  <p>\u201cI\u2019m prepared to pay $200 for those two red chairs, but 300 if you can deliver them tomorrow.\u201d<\/p>\n<\/blockquote>\n\n<p>This is a multi-dimensional since the price is also about the delivery time not only the goods.<\/p>\n\n<h2 id=\"single-vs-double-sided-auctions\">Single vs Double-Sided Auctions<\/h2>\n\n<p>Single-sided markets refer to either <strong>one buyer<\/strong> and many sellers (buy-side markets), or <strong>one seller<\/strong> and many buyers (sell-side markets). The latter is the thing we normally think of as an auction. On its opposite, double-sided markets refer to either many buyers and many sellers.<\/p>\n\n<p>For example, the stock market is a double-sided market, where there are many buyers and many sellers. The same can be said for the housing market. On the other hand, if we have a company that is the only buyer of a particular product or service, then we have a one buyer market.<\/p>\n\n<h2 id=\"open-cry-vs-sealed-bid-auctions\">Open-Cry vs. Sealed-Bid Auctions<\/h2>\n\n<p>In an <strong>open-cry auction<\/strong>, all traders <strong>announce<\/strong> their offers to <strong>each other<\/strong>, so bidders have <strong>more information<\/strong> about the market. This can be contrasted with a <strong>sealed-bid auction<\/strong>, where <strong>only the auctioneer<\/strong> sees the offers. In some auction forms, traders may even pay for preferential access to information. For example, in an open-cry auction, a bidder may be able to observe the offers made by other traders and adjust their own bid accordingly. In a sealed-bid auction, bidders do not have access to this information and must make their bids based on their own valuations.<\/p>\n\n<h2 id=\"single-vs-multi-unit-auctions\">Single vs. Multi-Unit Auctions<\/h2>\n\n<p>In a <strong>single-unit auction<\/strong>, only one unit of the same good is allowed to be bid for at a time. If there are multiple units to be sold, the auctioneer may repeat the bidding process until all units are sold. For example, in an English auction selling a house, each bidder can keep bidding until a highest bidder is found, and if the highest bidder does not reach the reserve price, the auctioneer may then start another round of bidding. In contrast, in a <strong>multi-unit auction<\/strong>, bidders can bid on both the price and quantity of the units they want. The auctioneer will then determine the successful bidders and the quantity each gets. Note that the <strong>unit<\/strong> refers to the <strong>indivisible unit<\/strong> that we are <strong>selling<\/strong>, such as a single fish versus a box of fish.<\/p>\n\n<h2 id=\"first-vs-k-th-price-auctions\">First vs K-th Price Auctions<\/h2>\n\n<p>The core difference between the two is how the price paid by the auction winner is determined. First-price auctions are auctions where the winner pays his or her own bid (i.e., the highest bid), and bidders need to <strong>weigh \u201cprobability of winning\u201d and \u201cprofit\u201d<\/strong>. The winner of the K-th highest-price auction pays the K-th highest bid (when K=2, it is the \u201csecond-highest-price auction\u201d), and bidders are motivated to <strong>bid according to the true valuation<\/strong>. Second-highest-price auctions are widely used in online advertising.<\/p>\n\n<h2 id=\"single-vs-multi-item-auctions\">Single vs. Multi-Item Auctions<\/h2>\n\n<p>A single-item auction is an auction where there is only one item for sale and the highest bidder wins the item. The valuation of the item is simple and direct. In contrast, a multi-item auction is an auction where there are multiple items for sale and bidders can bid on combinations of items. The valuation of the combination <strong>can be complex and may not be equal to the sum of the valuations of the individual items<\/strong>. There are two types of multi-item auctions: simultaneous auctions and combinatorial auctions. In simultaneous auctions, multiple items are auctioned off at the same time, and the allocation of the items is determined by complex algorithms. In combinatorial auctions, bidders can bid on all possible combinations of items, and the allocation is determined by the VCG mechanism (see <a href=\"#the-vcg-mechanism\">VCG<\/a>). <strong>The key difference between single-item and multi-item auctions is the complexity of the valuation of the items and the allocation of the items.<\/strong><\/p>\n\n<hr \/>\n\n<h1 id=\"standard-auction-types\">Standard Auction Types<\/h1>\n\n<p>By applying the above zoology, we can get a list of standard auction types which typically contains English Auctions, Dutch Auctions, First-Price Sealed Bid Auctions, Vickrey Auctions. We will start with English Auctions. And then in the next section we will move on to Combinatorial Auctions.<\/p>\n\n<h2 id=\"english-auctions\">English Auctions<\/h2>\n\n<table>\n  <thead>\n    <tr>\n      <th>Auction Type<\/th>\n      <th>Dimensionality<\/th>\n      <th>Sidedness<\/th>\n      <th>Units Available<\/th>\n      <th>Pricing Rule<\/th>\n      <th>Items Available<\/th>\n    <\/tr>\n  <\/thead>\n  <tbody>\n    <tr>\n      <td>Open-cry<\/td>\n      <td>Single<\/td>\n      <td>Single-sided<\/td>\n      <td>Single unit<\/td>\n      <td>First-price<\/td>\n      <td>Single item<\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n\n<p>An <strong>English auction<\/strong> is the most common type of auction, where a single item is sold to the highest bidder. The auction process typically starts with an initial bid, which is then <strong>increased by subsequent bidders<\/strong> until a final price is reached. In some cases, the auctioneer may call out prices and buyers indicate their acceptance of the price. The seller may set a reserve price, which is the lowest acceptable price. The auction ends when a <strong>fixed time is reached<\/strong> (e.g. in internet auctions) or when there is <strong>no more bidding<\/strong> activity. The last bidder pays their bid, which is the final price.<\/p>\n\n<h2 id=\"dutch-auctions\">Dutch Auctions<\/h2>\n\n<table>\n  <thead>\n    <tr>\n      <th>Auction Type<\/th>\n      <th>Dimensionality<\/th>\n      <th>Sidedness<\/th>\n      <th>Units Available<\/th>\n      <th>Pricing Rule<\/th>\n      <th>Items Available<\/th>\n    <\/tr>\n  <\/thead>\n  <tbody>\n    <tr>\n      <td>Open-cry<\/td>\n      <td>Single<\/td>\n      <td>Single-sided<\/td>\n      <td>Single unit<\/td>\n      <td>First-price<\/td>\n      <td>Single item<\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n\n<p>A Dutch auction, also known as a \u201cdescending clock\u201d auction, is a type of auction where the <strong>price starts high<\/strong> and is <strong>gradually lowered<\/strong> until a bidder accepts the current price. Some auctions use a clock to display the prices, and the auctioneer calls out descending prices until one bidder claims the good by indicating the current price is acceptable. If there are multiple bidders who are willing to pay the same price, the auctioneer restarts the descent from a slightly higher price than the tie occurred at. The winner pays the price at which they \u201cstop the clock\u201d.<\/p>\n\n<p>We can tell that since auction proceeds <strong>swiftly<\/strong>, Dutch auctions can be used to sell <strong>high volume<\/strong> in a short period of time. It is often used to sell <strong>perishable goods<\/strong> like, flowers in the Netherlands, fish in Spain, and tobacco in Canada.<\/p>\n\n<h2 id=\"first-price-sealed-bid-auctions\">First-Price Sealed Bid Auctions<\/h2>\n\n<table>\n  <thead>\n    <tr>\n      <th>Auction Type<\/th>\n      <th>Dimensionality<\/th>\n      <th>Sidedness<\/th>\n      <th>Units Available<\/th>\n      <th>Pricing Rule<\/th>\n      <th>Items Available<\/th>\n    <\/tr>\n  <\/thead>\n  <tbody>\n    <tr>\n      <td>Sealed-bid<\/td>\n      <td>Single<\/td>\n      <td>Single-sided<\/td>\n      <td>Single unit<\/td>\n      <td>First-price<\/td>\n      <td>Single item<\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n\n<p>In contrast to English auctions, where bidders can <strong>gain valuable information about the market from others\u2019 bids<\/strong>, sealed-bid auctions like the First-Price Sealed Bid (FPSB) auction do not provide such information. Instead, the highest bidder wins the auction and pays the price they bid, with <strong>no opportunity to adjust their bid based on others\u2019 valuations<\/strong>.<\/p>\n\n<p>Governments often use this mechanism to sell treasury bonds, e.g. UK still does while US recently changed to SPSB (Second Price Sealed Bid, a.k.a. Vickrey Auction). Property can also be sold this way as in Scotland.<\/p>\n\n<h2 id=\"vickrey-auctions\">Vickrey Auctions<\/h2>\n\n<table>\n  <thead>\n    <tr>\n      <th>Auction Type<\/th>\n      <th>Dimensionality<\/th>\n      <th>Sidedness<\/th>\n      <th>Units Available<\/th>\n      <th>Pricing Rule<\/th>\n      <th>Items Available<\/th>\n    <\/tr>\n  <\/thead>\n  <tbody>\n    <tr>\n      <td>Sealed-bid<\/td>\n      <td>Single<\/td>\n      <td>Single-sided<\/td>\n      <td>Single unit<\/td>\n      <td>Second-price<\/td>\n      <td>Single item<\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n\n<p>The Vickrey auction is a <strong>sealed-bid auction<\/strong> mechanism in which the winning bidder pays the amount of the <strong>second-highest bid<\/strong>. This design is incentive compatible, meaning that bidders have an incentive to bid their true value. However, it is not a panacea and has been known to have issues in practice, such as when the New Zealand government used it to auction off spectrum licenses.<\/p>\n\n<p>The Vickrey auction\u2019s incentive compatibility stems from the fact that <strong>bidding truthfully is a dominant strategy<\/strong>. If a bidder bids more than their valuation, they may win the good but risk paying more than they think it\u2019s worth. On the other hand, if they bid less than their valuation, they stand less chance of winning but may still end up paying the same price if they do win. Therefore, <strong>there is no point in bidding above or below one\u2019s true valuation<\/strong>, as it will not affect the outcome or the price paid.<\/p>\n\n<hr \/>\n\n<h1 id=\"combinatorial-auctions\">Combinatorial Auctions<\/h1>\n\n<p>Combinatorial auctions are particularly useful for allocating <strong>bundles of goods<\/strong>, such as spectrum licenses. For example, a telecommunications company may want to acquire licenses for the same bandwidth in different regions, such as Brooklyn, Manhattan, Queens, and Staten Island. In this case, the <strong>licenses for the same bandwidth are the most valuable, but a different bandwidth license is still more valuable than no license at all<\/strong>. The Federal Communications Commission (FCC) has used combinatorial auctions to allocate spectrum licenses in the past, but the auctions were not designed to handle bundles of goods.<\/p>\n\n<h2 id=\"notations-and-definitions\">Notations and Definitions<\/h2>\n\n<p>Before we go further, we need to define some notations and definitions. The set of items to be auctioned is denoted as:<\/p>\n\n\\[\\mathcal{Z} = \\{z_1, z_2, \\ldots, z_m\\}\\]\n\n<p>Then, we give the usual set of agents:<\/p>\n\n\\[\\mathcal{Ag} = \\{1, 2, \\ldots, n\\}\\]\n\n<p>Capture preferences of agent $i$ with the valuation function:<\/p>\n\n\\[v_i: 2^\\mathcal{Z} \\rightarrow \\mathbb{R}\\]\n\n<p>which means that for every possible bundle of goods $Z \\subseteq \\mathcal{Z}$, agent $i$ has a valuation $v_i(Z)$. To specify the extreme cases, $v_i(\\emptyset) = 0$.<\/p>\n\n<p>Another useful idea is <strong>free disposal<\/strong>, an agent is never worse off having more stuff.<\/p>\n\n\\[Z_1 \\subseteq Z_2 \\implies v_i(Z_1) \\leq v_i(Z_2)\\]\n\n<p>Formally an allocation is a list of sets $Z_1 , Z_2, \\ldots, Z_n$, one for each agent $Ag_i$ with the stipulation that:<\/p>\n\n\\[Z_i, Z_j \\subseteq \\mathcal{Z} \\quad Z_i \\cap Z_j = \\emptyset \\quad \\forall i \\neq j\\]\n\n<p>Thus no good is allocated to more than one agent. The set of all allocations of $Z$ to agents $Ag$ is denoted as:<\/p>\n\n\\[alloc(Z, Ag)\\]\n\n<p>If we design the auction, we get to say how the allocation is determined. One natural way is to maximize social welfare which is the sum of utilities of all the agents. The social welfare is denoted as:<\/p>\n\n\\[sw(Z_1, Z_2, \\ldots, Z_n;v_1, v_2, \\ldots, v_n) = \\sum_{i=1}^n v_i(Z_i)\\]\n\n<p>Given this, we can define a combinatorial auction. Given a set of goods $Z$ and a collection of valuation functions $v_1, v_2, \\ldots, v_n$, one for each agent $I \\in Ag$, the goal is to find an allocation:<\/p>\n\n\\[Z_1^*, Z_2^*, \\ldots, Z_n^* = \\text{argmax}_{(Z_1, Z_2, \\ldots, Z_n) \\in alloc(Z, Ag)} sw(Z_1, Z_2, \\ldots, Z_n;v_1, v_2, \\ldots, v_n)\\]\n\n<p>Figuring this out is winner determination. We can achieve this by having every agent $i$ to declare their valuation $\\hat{v}_i$. Then we just look at all the possible allocations and figure out what the best one is. However, on the one hand the hat denotes that this is what the agent <strong>says<\/strong>, not what it necessarily is. And on the other hand, the problem is representation, valuations are <strong>exponential<\/strong>:<\/p>\n\n\\[v_i: 2^\\mathcal{Z} \\rightarrow \\mathbb{R}\\]\n\n<p>which means that searching through is computationally intractable.<\/p>\n\n<h2 id=\"bidding-languages\">Bidding Languages<\/h2>\n\n<p>Rather than exhaustive evaluations, allow bidders to construct valuations from the bids they want to mention. Atomic bids are denoted as:<\/p>\n\n\\[(Z, p), \\quad Z \\subseteq \\mathcal{Z}, \\quad p \\in \\mathbb{R}\\]\n\n<p>A bundle $Z^\\prime$ <strong>satisfies<\/strong> a bid $(Z, p)$ if $Z \\subseteq Z^\\prime$. In other words a bundle satisfies a bid if it contains at least the things in the bid. Atomic bids define valuations as follows:<\/p>\n\n\\[v_\\beta(Z^\\prime) = \\begin{cases}\n        p &amp; \\text{if } Z^\\prime \\subseteq Z \\\\\n        0 &amp; \\text{otherwise}\n    \\end{cases}\\]\n\n<p>But atomic bids alone don\u2019t allow us to construct very interesting valuations. To construct more complex valuations, atomic bids can be combined into more complex bids. One approach is $XOR$ bids. Here we give an example of it:<\/p>\n\n\\[\\beta_1 = (\\{a, b\\}, 3) \\text{ XOR } (\\{b, c\\}, 5)\\]\n\n<p>which can be read to mean I would pay $3$ for a bundle that contains a and $b$ but not $c$ and $d$, and I will pay $5$ for a bundle that contains $c$ and $d$ but not $a$ and $b$, and I will pay $5$ for a bundle that contains $a$, $b$, $c$ and $d$. Constructing valuations:<\/p>\n\n\\[\\begin{align*}\n    v_{\\beta_1}(\\{a\\}) &amp;= 0\\\\\n    v_{\\beta_1}(\\{b\\}) &amp;= 0\\\\\n    v_{\\beta_1}(\\{a, b\\}) &amp;= 3\\\\\n    v_{\\beta_1}(\\{c, d\\}) &amp;= 5\\\\\n    v_{\\beta_1}(\\{a, b, c, d\\}) &amp;= 5\\\\\n\\end{align*}\\]\n\n<p>More formally, a bid like this:<\/p>\n\n\\[\\beta = (Z_1, p_1) \\text{ XOR } (Z_2, p_2) \\text{ XOR } \\ldots \\text{ XOR } (Z_n, p_n)\\]\n\n<p>Defines a valuation $v_\\beta$ as follows:<\/p>\n\n\\[v_\\beta(Z^\\prime) = \\begin{cases}\n        \\max \\{p_i | Z_i \\subseteq Z^\\prime \\} &amp; \\text{if } Z_i \\subseteq Z^\\prime \\\\\n        0 &amp; \\text{otherwise}\n    \\end{cases}\\]\n\n<p>XOR bids are <strong>fully expressive<\/strong>, that they can express any valuation function over a set of good. To do that, we may need an exponentially large number of atomic bids. However, the valuation of a bundle can be computed in polynomial time.<\/p>\n\n<h2 id=\"the-vcg-mechanism\">The VCG Mechanism<\/h2>\n\n<p>In general, we do not know whether the $\\hat{v}_i$ are the true valuations. Life would be easier if they were! In a generalization of the Vickrey auction (Vickrey\/Clarke\/Groves Mechanism). Mechanism is incentive compatible: telling the truth is a dominant strategy.<\/p>\n\n<p>To demonstrate this, we need to define more notations:<\/p>\n\n<p>Indifferent valuation function is that<\/p>\n\n\\[v^0(Z) = 0, \\quad \\forall Z \\subseteq \\mathcal{Z}\\]\n\n<p>$sw_{-i}$ is the social welfare function without $i$ is:<\/p>\n\n\\[sw_{-i}(Z_1, \\ldots, Z_n; \\hat{v}_1, \\ldots, \\hat{v}_n) = \\sum_{j \\neq i} v_j(Z_j)\\]\n\n<p>Now we can define the VCG mechanism:<\/p>\n\n<ul>\n  <li>Every agents simultaneously declares a valuation $\\hat{v}_i$.<\/li>\n  <li>\n    <p>The mechanism computes as followsm, and the allocation $Z_1^<em>, Z_2^<\/em>, \\ldots, Z_n^*$ is chosen.<\/p>\n\n\\[Z_1^*, \\ldots, Z_n^* = \\text{argmax}_{(Z_1, \\ldots, Z_n) \\in alloc(\\mathcal{Z}, \\mathcal{Ag})} sw(Z_1, \\ldots, Z_n; \\hat{v}_1, \\ldots, \\hat{v}_n)\\]\n  <\/li>\n  <li>\n    <p>The mechanism also computes, for each agent $i$:<\/p>\n\n\\[Z_1^\\prime, \\ldots, Z_n^\\prime = \\text{argmax}_{(Z_1, \\ldots, Z_n) \\in alloc(\\mathcal{Z}, \\mathcal{Ag})} sw(Z_1, \\ldots, Z_n; \\hat{v}_1, \\ldots, v^0, \\ldots, \\hat{v}_n)\\]\n  <\/li>\n  <li>\n    <p>Every agent $i$ pays $p_i$, where<\/p>\n\n\\[p_i = sw_{-i}(Z_1^\\prime, \\ldots, Z_n^\\prime; \\hat{v}_1, \\ldots, v^0, \\ldots, \\hat{v}_n) - sw_{-i}(Z_1^\\prime, \\ldots, Z_n^\\prime; \\hat{v}_1, \\ldots, \\hat{v}_n)\\]\n  <\/li>\n<\/ul>\n\n<p>In essence, each agent pays the cost to other agents for participating in the auction. This mechanism is <strong>incentive compatible<\/strong>, similar to the Vickrey auction. If an agent bids above their true valuation and wins, they end up paying what the good is worth to others, which exceeds its value to them. Conversely, if an agent bids conservatively, they reduce their chances of winning, yet still pay what others deem the good is worth, thus not saving any money. Consequently, a dominant strategy emerges for each agent, ensuring the maximization of social welfare.<\/p>\n\n<hr \/>\n\n<p>Related Posts \/ Websites \ud83d\udc47<\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Intelligent-Agent-Prolegomenon\">Ray - NTU SC4003 Lecture 3 Note: Intelligent Agent Prolegomenon<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Agent-Decision-Making\">Ray - NTU SC4003 Lecture 4 Note: Agent Decision Making<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Agent-Architecture\">Ray - NTU SC4003 Lecture 5 Note: Agent Architecture<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Working-Together-Benevolent-Cooperative-Agents\">Ray - NTU SC4003 Lecture 7 Note: Working Together Benevolent\/Cooperative Agents<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Self-Interested-Agents-Game-Theory-Foundation\">Ray - NTU SC4003 Lecture 8 Note: Game Theory Foundations<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Making-Group-Decisions-Voting\">Ray - NTU SC4003 Lecture 10 Note: Voting Mechanism<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Forming-Coalition\">Ray - NTU SC4003 Lecture 11 Note: Forming Coalition<\/a><\/p>\n","pubDate":"Thu, 03 Apr 2025 00:00:00 +0000","link":"https:\/\/huruilizhen.github.io\/Allocating-Scarce-Resources-Auction","guid":"https:\/\/huruilizhen.github.io\/Allocating-Scarce-Resources-Auction","category":"intelligent-agent"},{"title":"String Matching","description":"<p>String matching is a fundamental problem in computer science, which involves <strong>finding the first occurrence of a given pattern in a text<\/strong>. This problem arises in many applications, such as searching for a character string in a text, finding a pattern in DNA sequences, decoding graphical or audio data, or searching for a sublist in linked lists. In this post, we will explore three different approaches to solving the string matching problem: a straightforward solution, the Rabin-Karp Algorithm, and the Boyer-Moore Algorithm. Also, this post is a lecture notes of the <code class=\"language-plaintext highlighter-rouge\">SC2001<\/code> course in NTU, covering common string matching algorithms.<\/p>\n\n<!--more-->\n\n<p>Before we start, let\u2019s define some notations:<\/p>\n<ul>\n  <li>$T$ is the text string<\/li>\n  <li>$P$ is the pattern string<\/li>\n  <li>$n$ is the length of $T$, $j$ is the position index of $T$<\/li>\n  <li>$m$ is the length of $P$, $k$ is the position index of $P$<\/li>\n<\/ul>\n\n<h2 id=\"table-of-contents\">Table of Contents<\/h2>\n<ul>\n  <li><a href=\"#simplescan-algorithm\">SimpleScan Algorithm<\/a><\/li>\n  <li><a href=\"#rabin-karp-algorithm\">Rabin-Karp Algorithm<\/a>\n    <ul>\n      <li><a href=\"#rabin-karp-algorithm-with-hash-function\">Rabin-Karp Algorithm with Hash Function<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#boyer-moore-algorithm\">Boyer-Moore Algorithm<\/a>\n    <ul>\n      <li><a href=\"#preprocessing---charjump\">Preprocessing - CharJump<\/a><\/li>\n      <li><a href=\"#preprocessing---matchjump\">Preprocessing - MatchJump<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#conclusion\">Conclusion<\/a><\/li>\n<\/ul>\n\n<hr \/>\n\n<h1 id=\"simplescan-algorithm\">SimpleScan Algorithm<\/h1>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kt\">int<\/span> <span class=\"nf\">SimpleScan<\/span><span class=\"p\">(<\/span><span class=\"kt\">char<\/span> <span class=\"n\">P<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">char<\/span> <span class=\"n\">T<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">n<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">m<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kt\">int<\/span> <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>  <span class=\"c1\">\/\/ the current guess where P begins in T<\/span>\n    <span class=\"kt\">int<\/span> <span class=\"n\">j<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>  <span class=\"c1\">\/\/ the index of the current character of T<\/span>\n    <span class=\"kt\">int<\/span> <span class=\"n\">k<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>  <span class=\"c1\">\/\/ the index of the current character of P<\/span>\n\n    <span class=\"k\">while<\/span> <span class=\"p\">(<\/span><span class=\"n\">j<\/span> <span class=\"o\">&lt;<\/span> <span class=\"n\">n<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">T<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">]<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">P<\/span><span class=\"p\">[<\/span><span class=\"n\">k<\/span><span class=\"p\">])<\/span> <span class=\"p\">{<\/span>     <span class=\"c1\">\/\/ mismatch<\/span>\n            <span class=\"n\">j<\/span> <span class=\"o\">=<\/span> <span class=\"o\">++<\/span><span class=\"n\">i<\/span><span class=\"p\">;<\/span>            <span class=\"c1\">\/\/ skip to the next guess<\/span>\n            <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">i<\/span> <span class=\"o\">&gt;<\/span> <span class=\"n\">n<\/span> <span class=\"o\">-<\/span> <span class=\"n\">m<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>    <span class=\"c1\">\/\/ not have enough characters<\/span>\n                <span class=\"k\">break<\/span><span class=\"p\">;<\/span>\n            <span class=\"p\">}<\/span>\n            <span class=\"n\">k<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>              <span class=\"c1\">\/\/ reset the index of P<\/span>\n        <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>                <span class=\"c1\">\/\/ match<\/span>\n            <span class=\"n\">j<\/span><span class=\"o\">++<\/span><span class=\"p\">;<\/span>\n            <span class=\"n\">k<\/span><span class=\"o\">++<\/span><span class=\"p\">;<\/span>\n            <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">k<\/span> <span class=\"o\">==<\/span> <span class=\"n\">m<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>       <span class=\"c1\">\/\/ found<\/span>\n                <span class=\"k\">return<\/span> <span class=\"n\">i<\/span><span class=\"p\">;<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">return<\/span> <span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>Comparison starts with $k=0$ and $j=i$. When $k$ reaches $m$, all characters have been compared and matched.<\/p>\n\n<p>When a mismatch happens, shift the pattern right one position: <code class=\"language-plaintext highlighter-rouge\">j=++i, k=0<\/code><\/p>\n\n<p>The worst case happens when the pattern appears right at the end of the string, and each character in the text must be compared with each character in the pattern. This results in a time complexity of $O(mn)$, where $m$ and $n$ are the lengths of the pattern and the text respectively. For example, if the pattern is \u201caaaa\u201d and the text is \u201caaaaaac\u201d.<\/p>\n\n<p>Furthermore, we can eliminate variable $i$ by using the fact $i = j - k$. This modification also mentioned in the tutorial.<\/p>\n\n<hr \/>\n\n<h1 id=\"rabin-karp-algorithm\">Rabin-Karp Algorithm<\/h1>\n\n<p>To tell two strings are the same or not is much harder than telling two integers are the same or not. The main idea of Rabin-Karp Algorithm is to convert the given string slice into an integer.<\/p>\n\n<p>The ourline of the <strong>Rabin-Karp Algorithm<\/strong> can be discribed as follows:<\/p>\n<ol>\n  <li>Convert the pattern (length $m$) to a number $p$<\/li>\n  <li>Convert the first $m$-characters (the first text window) to a number $t$<\/li>\n  <li>If $p$ and $t$ are equal, pattern found and exit<\/li>\n  <li>If not end-of-text, shift the text window one character right and convert the string in it to a number t, go to step $3$; else pattern not found and exit<\/li>\n<\/ol>\n\n<p>To compute the number for the given string slice, we have the following notations:<\/p>\n\n<ul>\n  <li>$\\Sigma$ is the set of all possible characters in the context. e.g. $\\Sigma = {a, b, c, d}$<\/li>\n  <li>$d = |\\Sigma|$ is the number of characters in $\\Sigma$ or the size of the alphabet<\/li>\n<\/ul>\n\n<p>The number $p$ of the pattern and the number t of the first $m$-character text window, are calculated iteratively. Take the example of the pattern $P$ \u201c36451\u201d and $d = 10$. The value of $p$ can be calculated as follows:<\/p>\n\n\\[\\begin{aligned}\np &amp;= 1 \\times 10^0 + 5 \\times 10^1 + 4 \\times 10^2 + 6 \\times 10^3 + 3 \\times 10^4 \\\\\n&amp;= 1 + 50 + 400 + 6000 + 30000 = 36451\n\\end{aligned}\\]\n\n<p>Or we can do it recursively:<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kt\">int<\/span> <span class=\"n\">p<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>\n<span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kt\">int<\/span> <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span> <span class=\"n\">i<\/span> <span class=\"o\">&lt;<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span> <span class=\"n\">i<\/span><span class=\"o\">++<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">p<\/span> <span class=\"o\">=<\/span> <span class=\"n\">p<\/span> <span class=\"o\">*<\/span> <span class=\"n\">d<\/span> <span class=\"o\">+<\/span> <span class=\"p\">(<\/span><span class=\"n\">P<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span> <span class=\"o\">-<\/span> <span class=\"sc\">'0'<\/span><span class=\"p\">);<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>The calculation can be done in linear time.<\/p>\n\n<p>To compute the number t after shifting the text window, it can be done in constant time based on the number of the previous text window. In general:<\/p>\n\n\\[\\text{new} = (\\text{old} - \\text{old's first character value} \\times d^{m-1}) * d + \\text{new's last character value}\\]\n\n<p>$d^{m-1}$ here can be precomputed and stored, so the calculation can be done in constant time.<\/p>\n\n<h2 id=\"rabin-karp-algorithm-with-hash-function\">Rabin-Karp Algorithm with Hash Function<\/h2>\n\n<p>But when we have a long pattern and a long text, the calculation of $p$ will likely cause overflow. To avoid this, we <strong>hash<\/strong> the value by taking it mod a prime number $q$. And this prime number should be large. We can reorganize the steps as follows:<\/p>\n\n<ol>\n  <li>Hash the pattern to a number, $p_h$<\/li>\n  <li>Hash the first m-character text window to a number, $t_h$<\/li>\n  <li>If $p_h$ and $t_h$ are equal, compare the pattern with the text window. If equal, pattern found and exit<\/li>\n  <li>If not end-of-text, shift the text window one character right and (re)hash it to a number $t_h$, go to step 3; else pattern not found and exit<\/li>\n<\/ol>\n\n<p>Note the fact that if $p_h  = t_h$ not necessarily mean the pattern is found. But if $p_h \\neq t_h$, the pattern is definitely not found.<\/p>\n\n<p>To calculate value of a given string slice, we need to use a hash function $hash(txt, m, d)$:<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kt\">int<\/span> <span class=\"nf\">hash<\/span><span class=\"p\">(<\/span><span class=\"kt\">int<\/span> <span class=\"n\">txt<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">m<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">d<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kt\">int<\/span> <span class=\"n\">h<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>\n    <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kt\">int<\/span> <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span> <span class=\"n\">i<\/span> <span class=\"o\">&lt;<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span> <span class=\"n\">i<\/span><span class=\"o\">++<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">h<\/span> <span class=\"o\">=<\/span> <span class=\"n\">h<\/span> <span class=\"o\">*<\/span> <span class=\"n\">d<\/span> <span class=\"o\">+<\/span> <span class=\"n\">txt<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">];<\/span>\n        <span class=\"n\">h<\/span> <span class=\"o\">%=<\/span> <span class=\"n\">q<\/span><span class=\"p\">;<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">h<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>where $q$ is a large prime number, txt is the converted string slice (integer array), and $d$ is the size of the alphabet.<\/p>\n\n<p>And to calculate the hash value of the next text window, we need to use a hash function $rehash(h, c, m, d)$:<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kt\">int<\/span> <span class=\"nf\">rehash<\/span><span class=\"p\">(<\/span><span class=\"kt\">int<\/span> <span class=\"n\">h<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">old_char<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">new_char<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">m<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">d<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">h<\/span> <span class=\"o\">=<\/span> <span class=\"p\">(<\/span><span class=\"n\">h<\/span> <span class=\"o\">-<\/span> <span class=\"n\">old_char<\/span> <span class=\"o\">*<\/span> <span class=\"n\">d<\/span><span class=\"o\">^<\/span><span class=\"p\">(<\/span><span class=\"n\">m<\/span><span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"p\">))<\/span> <span class=\"o\">*<\/span> <span class=\"n\">d<\/span> <span class=\"o\">+<\/span> <span class=\"n\">new_char<\/span><span class=\"p\">;<\/span>\n    <span class=\"n\">h<\/span> <span class=\"o\">%=<\/span> <span class=\"n\">q<\/span><span class=\"p\">;<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">h<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>where $h$ is the hash value of the previous text window, $old_char$ is the first character of the previous text window, $new_char$ is the last character of the new text window, and $d$ is the size of the alphabet. <code class=\"language-plaintext highlighter-rouge\">d^(m-1)<\/code> can be precomputed and stored.<\/p>\n\n<p>The whole Rabin-Karp Algorithm can be implemented as follows:<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kt\">int<\/span> <span class=\"nf\">RKscan<\/span><span class=\"p\">(<\/span><span class=\"kt\">char<\/span> <span class=\"n\">P<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">char<\/span> <span class=\"n\">T<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">n<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">m<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kt\">int<\/span> <span class=\"n\">p<\/span> <span class=\"o\">=<\/span> <span class=\"n\">hash<\/span><span class=\"p\">(<\/span><span class=\"n\">P<\/span><span class=\"p\">,<\/span> <span class=\"n\">m<\/span><span class=\"p\">,<\/span> <span class=\"n\">d<\/span><span class=\"p\">);<\/span>\n    <span class=\"kt\">int<\/span> <span class=\"n\">t<\/span> <span class=\"o\">=<\/span> <span class=\"n\">hash<\/span><span class=\"p\">(<\/span><span class=\"n\">T<\/span><span class=\"p\">,<\/span> <span class=\"n\">m<\/span><span class=\"p\">,<\/span> <span class=\"n\">d<\/span><span class=\"p\">);<\/span>\n    <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kt\">int<\/span> <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span> <span class=\"n\">i<\/span> <span class=\"o\">&lt;=<\/span> <span class=\"n\">n<\/span> <span class=\"o\">-<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span> <span class=\"n\">i<\/span><span class=\"o\">++<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">p<\/span> <span class=\"o\">==<\/span> <span class=\"n\">t<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n            <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">strcmp<\/span><span class=\"p\">(<\/span><span class=\"n\">P<\/span><span class=\"p\">,<\/span> <span class=\"n\">T<\/span> <span class=\"o\">+<\/span> <span class=\"n\">i<\/span><span class=\"p\">)<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">0<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n                <span class=\"k\">return<\/span> <span class=\"n\">i<\/span><span class=\"p\">;<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">}<\/span>\n        <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">i<\/span> <span class=\"o\">&lt;<\/span> <span class=\"n\">n<\/span> <span class=\"o\">-<\/span> <span class=\"n\">m<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n            <span class=\"n\">t<\/span> <span class=\"o\">=<\/span> <span class=\"n\">rehash<\/span><span class=\"p\">(<\/span><span class=\"n\">t<\/span><span class=\"p\">,<\/span> <span class=\"n\">T<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">],<\/span> <span class=\"n\">T<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span> <span class=\"o\">+<\/span> <span class=\"n\">m<\/span><span class=\"p\">],<\/span> <span class=\"n\">m<\/span><span class=\"p\">,<\/span> <span class=\"n\">d<\/span><span class=\"p\">);<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">return<\/span> <span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"p\">;<\/span>    \n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>The running time of Rabin-Karp algorithm in the worst case is $\\theta((n - m + 1)m)$. However, in many applications, the expected running time is $O(n + m)$ plus the time required to process spurious hits. The algorithm requires $O(m)$ time for the two <code class=\"language-plaintext highlighter-rouge\">hash()<\/code> calls and close to $O(n)$ time on the for loop. The number of spurious hits can be kept low by using a large prime number $q$ for the hash functions.<\/p>\n\n<hr \/>\n\n<h1 id=\"boyer-moore-algorithm\">Boyer-Moore Algorithm<\/h1>\n\n<p>The Boyer-Moore Algorithm is a very efficient string searching algorithm. It processes the text being scanned, $T$, from left to right, and the pattern we are looking for, $P$, from right to left. To optimize the search, <strong>two tables<\/strong> are generated during preprocessing, which allow us to <strong>slide the pattern as far as possible<\/strong> after a mismatch. This algorithm is particularly efficient for long patterns.<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kt\">int<\/span> <span class=\"nf\">BMscan<\/span><span class=\"p\">(<\/span><span class=\"kt\">char<\/span> <span class=\"n\">P<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">char<\/span> <span class=\"n\">T<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">n<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">m<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">charJump<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">matchJump<\/span><span class=\"p\">[])<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kt\">int<\/span> <span class=\"n\">k<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span>  <span class=\"c1\">\/\/ the index of the current character of P, assume the string starts from index 1<\/span>\n    <span class=\"kt\">int<\/span> <span class=\"n\">j<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span>  <span class=\"c1\">\/\/ the index of the current character of T, assume the string starts from index 1<\/span>\n    <span class=\"k\">while<\/span> <span class=\"p\">(<\/span><span class=\"n\">j<\/span> <span class=\"o\">&lt;=<\/span> <span class=\"n\">n<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">k<\/span> <span class=\"o\">&lt;=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">)<\/span>\n            <span class=\"k\">return<\/span> <span class=\"n\">j<\/span> <span class=\"o\">+<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span> <span class=\"c1\">\/\/ match found<\/span>\n        <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">P<\/span><span class=\"p\">[<\/span><span class=\"n\">k<\/span><span class=\"p\">]<\/span> <span class=\"o\">==<\/span> <span class=\"n\">T<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">])<\/span> <span class=\"p\">{<\/span>\n            <span class=\"n\">k<\/span><span class=\"o\">--<\/span><span class=\"p\">;<\/span>\n            <span class=\"n\">j<\/span><span class=\"o\">--<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>            <span class=\"c1\">\/\/ mismatch<\/span>\n            <span class=\"n\">j<\/span> <span class=\"o\">+=<\/span> <span class=\"n\">max<\/span><span class=\"p\">(<\/span><span class=\"n\">charJump<\/span><span class=\"p\">[<\/span><span class=\"n\">T<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">]],<\/span> <span class=\"n\">matchJump<\/span><span class=\"p\">[<\/span><span class=\"n\">k<\/span> <span class=\"o\">+<\/span> <span class=\"mi\">1<\/span><span class=\"p\">]);<\/span>\n            <span class=\"n\">k<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">return<\/span> <span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p><code class=\"language-plaintext highlighter-rouge\">charJump<\/code> and <code class=\"language-plaintext highlighter-rouge\">matchJump<\/code> are the 2 tables generated in a preprocessing step.<\/p>\n\n<h2 id=\"preprocessing---charjump\">Preprocessing - CharJump<\/h2>\n\n<p>The core idea of the <code class=\"language-plaintext highlighter-rouge\">charJump<\/code> table is to compute the maximum number of characters to skip when a mismatch occurs by <strong>aligning the mismatched character with the rightmost occurrence<\/strong> of the same character in the pattern. Detailly:<\/p>\n\n<ul>\n  <li>If $T_j$ does not appear in $P$ at all, we line up $P$ after $T_j$<\/li>\n  <li>If $T_j$ occurs in $P$, we line up $T_j$ with the rightmost instance of $T_j$ in $P$<\/li>\n<\/ul>\n\n<p>It is quite simple to compute the <code class=\"language-plaintext highlighter-rouge\">charJump<\/code> table, the time complexity is $O(m + |\\Sigma|)$.<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kt\">void<\/span> <span class=\"nf\">computeCharJump<\/span><span class=\"p\">(<\/span><span class=\"kt\">char<\/span> <span class=\"n\">P<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">m<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">charJump<\/span><span class=\"p\">[])<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kt\">char<\/span> <span class=\"n\">ch<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span> <span class=\"n\">ch<\/span> <span class=\"o\">&lt;<\/span> <span class=\"n\">alpha<\/span><span class=\"p\">;<\/span> <span class=\"n\">ch<\/span><span class=\"o\">++<\/span><span class=\"p\">)<\/span>\n        <span class=\"n\">charJump<\/span><span class=\"p\">[<\/span><span class=\"n\">ch<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span>\n    <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kt\">int<\/span> <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span> <span class=\"n\">i<\/span> <span class=\"o\">&lt;=<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span> <span class=\"n\">i<\/span><span class=\"o\">++<\/span><span class=\"p\">)<\/span>\n        <span class=\"n\">charJump<\/span><span class=\"p\">[<\/span><span class=\"n\">P<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span> <span class=\"o\">-<\/span> <span class=\"n\">i<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>where <code class=\"language-plaintext highlighter-rouge\">alpha<\/code> is the size of the alphabet, and $m - i$ is the length from the current character position to the end of $P$. Notice that if a character appears more than once, we take the right-most occurrence.<\/p>\n\n<p>Sometimes this heuristic fails, when <code class=\"language-plaintext highlighter-rouge\">charJump<\/code> table gives a shorter jump than the length of the successful match $m - k$. In this case, we actually jump forward. Hence we need to choose the maximum of the two <code class=\"language-plaintext highlighter-rouge\">max(charJump[T[j]], m - k + 1)<\/code>. In this way, simplified BM (only <code class=\"language-plaintext highlighter-rouge\">charJump<\/code>) algorithm can be implemented:<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kt\">int<\/span> <span class=\"nf\">BMscan<\/span><span class=\"p\">(<\/span><span class=\"kt\">char<\/span> <span class=\"n\">P<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">char<\/span> <span class=\"n\">T<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">n<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">m<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">charJump<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">matchJump<\/span><span class=\"p\">[])<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kt\">int<\/span> <span class=\"n\">k<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span>  <span class=\"c1\">\/\/ the index of the current character of P, assume the string starts from index 1<\/span>\n    <span class=\"kt\">int<\/span> <span class=\"n\">j<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span>  <span class=\"c1\">\/\/ the index of the current character of T, assume the string starts from index 1<\/span>\n    <span class=\"k\">while<\/span> <span class=\"p\">(<\/span><span class=\"n\">j<\/span> <span class=\"o\">&lt;=<\/span> <span class=\"n\">n<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">k<\/span> <span class=\"o\">&lt;=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">)<\/span>\n            <span class=\"k\">return<\/span> <span class=\"n\">j<\/span> <span class=\"o\">+<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span>   <span class=\"c1\">\/\/ match found<\/span>\n        <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">P<\/span><span class=\"p\">[<\/span><span class=\"n\">k<\/span><span class=\"p\">]<\/span> <span class=\"o\">==<\/span> <span class=\"n\">T<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">])<\/span> <span class=\"p\">{<\/span>\n            <span class=\"n\">k<\/span><span class=\"o\">--<\/span><span class=\"p\">;<\/span>\n            <span class=\"n\">j<\/span><span class=\"o\">--<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>            <span class=\"c1\">\/\/ mismatch<\/span>\n            <span class=\"n\">j<\/span> <span class=\"o\">+=<\/span> <span class=\"n\">max<\/span><span class=\"p\">(<\/span><span class=\"n\">charJump<\/span><span class=\"p\">[<\/span><span class=\"n\">T<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">]],<\/span> <span class=\"n\">m<\/span> <span class=\"o\">-<\/span> <span class=\"n\">k<\/span> <span class=\"o\">+<\/span> <span class=\"mi\">1<\/span><span class=\"p\">);<\/span>\n            <span class=\"n\">k<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">return<\/span> <span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>In some resources, the principal of <code class=\"language-plaintext highlighter-rouge\">charJump<\/code> table is called <strong>bad character rule<\/strong>.<\/p>\n\n<h2 id=\"preprocessing---matchjump\">Preprocessing - MatchJump<\/h2>\n\n<p>This heuristic tries to derive the maximum shift from the structure of the pattern. It is defined for each of the characters (every position) in $P$. And we can roughly divide all possible situations into 3 cases:<\/p>\n\n<p><strong>Case 1<\/strong>: The matching suffix occurs earlier in the pattern, but preceded by a different character.<\/p>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/match-jump-case-1.png\" alt=\"MatchJump Case 1\" \/>\n  \n  <figcaption class=\"caption-text\">match jump case 1<\/figcaption>\n  \n<\/figure>\n\n<p>We line up the earlier occurrence of the suffix in $P$ with the matched substring in $T$.<\/p>\n\n<p><strong>Case 2<\/strong>: Only part of the matching suffix occurs at the beginning of the pattern (a prefix)<\/p>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/match-jump-case-2.png\" alt=\"MatchJump Case 2\" \/>\n  \n  <figcaption class=\"caption-text\">match jump case 2<\/figcaption>\n  \n<\/figure>\n\n<p>We line up the prefix in $P$ with part of the matched substring in $T$.<\/p>\n\n<p><strong>Case 3<\/strong>: There is no other occurrence of the matching suffix in the pattern. (Case 1 and Case 2 do not happen)<\/p>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/match-jump-case-3.png\" alt=\"MatchJump Case 3\" \/>\n  \n  <figcaption class=\"caption-text\">match jump case 3<\/figcaption>\n  \n<\/figure>\n\n<p>We line up $P$ after the matched substring in $T$.<\/p>\n\n<p>In both cases, we can compute the <code class=\"language-plaintext highlighter-rouge\">matchJump<\/code> value as follows:<\/p>\n\n\\[\\text{matchJump}(k) = \\text{length of the repeated suffix in } P_{k, m} + \\text{slide}(k)\\]\n\n<p>where $P_{k, m}$ is the suffix of $P$ starting at position $k$ and length $m - k + 1$, and <code class=\"language-plaintext highlighter-rouge\">slide<\/code> is the distance from the end (position $m$) to the end of the matched repeated suffix.<\/p>\n\n<p>From the above analysis, before we compute <code class=\"language-plaintext highlighter-rouge\">matchJump<\/code>, we need to compute two helpers array namely <code class=\"language-plaintext highlighter-rouge\">suffix<\/code> and <code class=\"language-plaintext highlighter-rouge\">prefix<\/code>. And definitions of the two helpers array are:<\/p>\n\n<ul>\n  <li>$\\text{suffix}(k)$: The position of the first occurrence of the suffix of length $k$. $\\text{suffix}(k) = j \\implies P_{j, j + k - 1} = P_{m - k + 1, m}$<\/li>\n  <li>$\\text{prefix}(k)$: Is the suffix of length k a prefix of the pattern string? $\\text{prefix}(k) = \\text{true} \\implies P_{1, k} = P_{m - k + 1, m}$<\/li>\n<\/ul>\n\n<p>To compute the helpers array, we can refer to the <code class=\"language-plaintext highlighter-rouge\">KMP<\/code> algorithm. Or we can do it naively by scanning the pattern string from the end to the beginning:<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kt\">void<\/span> <span class=\"nf\">computeHelpers<\/span><span class=\"p\">(<\/span><span class=\"kt\">char<\/span> <span class=\"n\">P<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">m<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">suffix<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">prefix<\/span><span class=\"p\">[])<\/span> <span class=\"p\">{<\/span>\n    <span class=\"c1\">\/\/ initialize suffix and prefix<\/span>\n    <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kt\">int<\/span> <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span> <span class=\"n\">i<\/span> <span class=\"o\">&lt;=<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span> <span class=\"o\">++<\/span><span class=\"n\">i<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">suffix<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>\n        <span class=\"n\">prefix<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">false<\/span><span class=\"p\">;<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kt\">int<\/span> <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span> <span class=\"n\">i<\/span> <span class=\"o\">&lt;<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span> <span class=\"o\">++<\/span><span class=\"n\">i<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"kt\">int<\/span> <span class=\"n\">j<\/span> <span class=\"o\">=<\/span> <span class=\"n\">i<\/span><span class=\"p\">;<\/span>  <span class=\"c1\">\/\/ the index of the first character to compare<\/span>\n        <span class=\"kt\">int<\/span> <span class=\"n\">k<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span>  <span class=\"c1\">\/\/ the length of the current matched suffix<\/span>\n\n        <span class=\"c1\">\/\/ match the suffix as far forward as possible<\/span>\n        <span class=\"k\">while<\/span> <span class=\"p\">(<\/span><span class=\"n\">j<\/span> <span class=\"o\">&gt;=<\/span> <span class=\"mi\">1<\/span> <span class=\"o\">&amp;&amp;<\/span> <span class=\"n\">P<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">]<\/span> <span class=\"o\">==<\/span> <span class=\"n\">P<\/span><span class=\"p\">[<\/span><span class=\"n\">m<\/span> <span class=\"o\">-<\/span> <span class=\"n\">k<\/span> <span class=\"o\">+<\/span> <span class=\"mi\">1<\/span><span class=\"p\">])<\/span> <span class=\"p\">{<\/span>\n            <span class=\"n\">suffix<\/span><span class=\"p\">[<\/span><span class=\"n\">k<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">j<\/span><span class=\"p\">;<\/span>\n            <span class=\"n\">j<\/span><span class=\"o\">--<\/span><span class=\"p\">;<\/span>\n            <span class=\"n\">k<\/span><span class=\"o\">++<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}<\/span>\n        <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">j<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">1<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>    <span class=\"c1\">\/\/ if the index goes at the bound, the current suffix is a prefix<\/span>\n            <span class=\"n\">prefix<\/span><span class=\"p\">[<\/span><span class=\"n\">k<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">true<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>Then we can compute the <code class=\"language-plaintext highlighter-rouge\">matchJump<\/code> table as follows:<\/p>\n\n<div class=\"language-cpp highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kt\">void<\/span> <span class=\"nf\">computeMatchJump<\/span><span class=\"p\">(<\/span><span class=\"kt\">int<\/span> <span class=\"n\">m<\/span><span class=\"p\">,<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">suffix<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">prefix<\/span><span class=\"p\">[],<\/span> <span class=\"kt\">int<\/span> <span class=\"n\">matchJump<\/span><span class=\"p\">[])<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kt\">int<\/span> <span class=\"n\">k<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span> <span class=\"n\">k<\/span> <span class=\"o\">&lt;<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span> <span class=\"o\">++<\/span><span class=\"n\">k<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span> \n        <span class=\"kt\">int<\/span> <span class=\"n\">j<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span> <span class=\"o\">-<\/span> <span class=\"n\">k<\/span> <span class=\"o\">+<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span>\n        <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">suffix<\/span><span class=\"p\">[<\/span><span class=\"n\">k<\/span><span class=\"p\">])<\/span> <span class=\"p\">{<\/span> <span class=\"c1\">\/\/ case 1<\/span>\n            <span class=\"n\">matchJump<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span> <span class=\"o\">-<\/span> <span class=\"n\">suffix<\/span><span class=\"p\">[<\/span><span class=\"n\">k<\/span><span class=\"p\">]<\/span> <span class=\"o\">+<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>\n            <span class=\"c1\">\/\/ case 3<\/span>\n            <span class=\"n\">matchJump<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span> <span class=\"o\">+<\/span> <span class=\"n\">k<\/span> <span class=\"o\">-<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span>\n            <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kt\">int<\/span> <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"n\">j<\/span> <span class=\"o\">+<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span> <span class=\"n\">i<\/span> <span class=\"o\">&lt;=<\/span> <span class=\"n\">m<\/span><span class=\"p\">;<\/span> <span class=\"o\">++<\/span><span class=\"n\">i<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n                <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"n\">prefix<\/span><span class=\"p\">[<\/span><span class=\"n\">m<\/span> <span class=\"o\">-<\/span> <span class=\"n\">i<\/span> <span class=\"o\">+<\/span> <span class=\"mi\">1<\/span><span class=\"p\">])<\/span> <span class=\"p\">{<\/span> <span class=\"c1\">\/\/ case 2<\/span>\n                    <span class=\"n\">matchJump<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">m<\/span> <span class=\"o\">+<\/span> <span class=\"n\">k<\/span> <span class=\"o\">-<\/span> <span class=\"p\">(<\/span><span class=\"n\">m<\/span> <span class=\"o\">-<\/span> <span class=\"n\">i<\/span> <span class=\"o\">+<\/span> <span class=\"mi\">1<\/span><span class=\"p\">);<\/span>\n                    <span class=\"k\">break<\/span><span class=\"p\">;<\/span>\n                <span class=\"p\">}<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>Also, the principal of <code class=\"language-plaintext highlighter-rouge\">matchJump<\/code> table is called <strong>good suffix rule<\/strong>.<\/p>\n\n<h1 id=\"conclusion\">Conclusion<\/h1>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/string-matching-conclusion.png\" alt=\"String Matching Conclusion\" \/>\n  \n  <figcaption class=\"caption-text\">string matching conclusion<\/figcaption>\n  \n<\/figure>\n\n<ul>\n  <li>Brute Force performed better than expected because the worst-case scenario is rare, occurring when the pattern and text nearly match.<\/li>\n  <li>Rabin-Karp performed worse due to <strong>expensive function calls<\/strong>, costly <strong>division operations<\/strong>, and the time-consuming <strong>conversion<\/strong> of character values to numeric values.<\/li>\n  <li>The <strong>Boyer-Moore<\/strong> algorithm is generally the <strong>most efficient string-matching algorithm<\/strong>, particularly in text editors. Moore notes that the algorithm tends to perform faster with longer patterns.<\/li>\n  <li>For binary strings, the <strong>Knuth-Morris-Pratt algorithm<\/strong> is recommended.<\/li>\n  <li>For very short patterns, the brute force algorithm may be more effective.<\/li>\n  <li>Insights from the BM algorithm: Solving problems often requires a deep understanding of the problem\u2019s <strong>structure<\/strong>. Analyze the problem thoroughly before devising a solution.<\/li>\n<\/ul>\n\n<p>In my previous algorithm competition experience, I learned about the <strong>KMP algorithm<\/strong> and <strong>AC automaton<\/strong> on the topic of string matching. Maybe I will summarize the blogs about these two algorithms later.<\/p>\n\n<hr \/>\n\n<p>Related Posts \/ Websites \ud83d\udc47<\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/www.geeksforgeeks.org\/boyer-moore-algorithm-for-pattern-searching\/\">geeksforgeeks - boyer moore algorithm for pattern searching<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"https:\/\/www.geeksforgeeks.org\/boyer-moore-algorithm-good-suffix-heuristic\/\">geeksforgeeks - booyer moore algorithm good suffix heuristic<\/a><\/p>\n","pubDate":"Wed, 02 Apr 2025 00:00:00 +0000","link":"https:\/\/huruilizhen.github.io\/String-Matching","guid":"https:\/\/huruilizhen.github.io\/String-Matching","category":"algorithm-design"},{"title":"Multiagent Decision Making among Self-Interested Agents: Game Theory Foundations","description":"<p>As we delve into the realm of multiagent decision making, game theory emerges as a powerful tool for understanding strategic interactions between self-interested agents. Born from the intersection of economics, mathematics, and computer science, game theory provides a robust framework for analyzing complex decision-making processes. In this post, we\u2019ll embark on an exploration of the game theory foundations that underlie the intricate dance of cooperation and competition among self-interested agents. This journey will cover the essential concepts and theories that form the backbone of Lecture 8 in the <code class=\"language-plaintext highlighter-rouge\">SC4003<\/code> course at NTU. Buckle up, and let\u2019s dive into the fascinating world of game theory!<\/p>\n\n<!--more-->\n\n<h2 id=\"table-of-contents\">Table of Contents<\/h2>\n<ul>\n  <li><a href=\"#notations-and-definitions\">Notations and Definitions<\/a>\n    <ul>\n      <li><a href=\"#utilities-and-preferences\">Utilities and Preferences<\/a><\/li>\n      <li><a href=\"#multiagent-encounters\">Multiagent Encounters<\/a><\/li>\n      <li><a href=\"#rational-action\">Rational Action<\/a><\/li>\n      <li><a href=\"#zero-sum-game-and-nonzero-sum-game\">Zero-Sum Game and Nonzero-Sum Game<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#solution-concepts\">Solution Concepts<\/a>\n    <ul>\n      <li><a href=\"#dominant-strategies\">Dominant Strategies<\/a><\/li>\n      <li><a href=\"#nash-equilibrium-strategies\">Nash Equilibrium Strategies<\/a><\/li>\n      <li><a href=\"#pareto-optimality-strategies\">Pareto Optimality Strategies<\/a><\/li>\n      <li><a href=\"#max-social-welfare-strategies\">Max Social Welfare Strategies<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#the-prisoners-dilemma\">The Prisoner\u2019s Dilemma<\/a><\/li>\n<\/ul>\n\n<hr \/>\n\n<h1 id=\"notations-and-definitions\">Notations and Definitions<\/h1>\n\n<p>Before we start, let\u2019s have a short definition of game theory notations, which will make the rest of this post easier to understand.<\/p>\n\n<h2 id=\"utilities-and-preferences\">Utilities and Preferences<\/h2>\n\n<p>Say we have two agents:<\/p>\n\n\\[Ag = \\{i, j\\}\\]\n\n<p>They are assumed to be <strong>self-interested<\/strong>, which means they have preferences over how the environment is. The <strong>set of \u201coutcomes\u201d<\/strong> that agents have preferences over is denoted as:<\/p>\n\n\\[\\Omega = \\{\\omega_1, \\omega_2, \\ldots, \\omega_n\\}\\]\n\n<p>We capture preferences by <strong>utility functions<\/strong>:<\/p>\n\n\\[U_i: \\Omega \\to \\mathbb{R}, \\quad U_j: \\Omega \\to \\mathbb{R}\\]\n\n<p>Utility functions lead to preference <strong>orderings<\/strong> over outcomes:<\/p>\n\n\\[\\begin{align*}\n    \\omega \\succ_i \\omega' &amp;\\iff U_i(\\omega) &gt; U_i(\\omega') \\\\\n    \\omega \\succeq_i \\omega' &amp;\\iff U_i(\\omega) \\geq U_i(\\omega') \\\\\n\\end{align*}\\]\n\n<p>Note that utility is not money, it is a <strong>measure of desirability<\/strong>. But it is a useful analogy.<\/p>\n\n<h2 id=\"multiagent-encounters\">Multiagent Encounters<\/h2>\n\n<p>We need a model of the environment in which these agents will act. Agents simultaneously choose an action in set $Ac$ to perform, and as a result of the actions they select, an outcome $\\omega$ in $\\Omega$ will result. The actual outcome depends on the <strong>combination of actions<\/strong>. Assume each agent has just two possible actions that it can perform, cooperate $C$ and defect $D$. Environment behavior given by state transformer function:<\/p>\n\n\\[\\tau: Ac \\times Ac \\to \\Omega\\]\n\n<p>For convenience, we can also express <strong>utility functions<\/strong> as follows. In fact, this notation will be used throughout this post instead of the one above.<\/p>\n\n\\[U_{agent}: Ac \\times Ac \\to \\mathbb{R}\\]\n\n<p>Classically, we will encounter following situation:<\/p>\n\n<ul>\n  <li>$\\tau(C, C) = \\omega_1$, $\\tau(C, D) = \\omega_2$, $\\tau(D, C) = \\omega_3$, $\\tau(D, D) = \\omega_4$: the environment is sensitive to actions of both agents.<\/li>\n  <li>$\\tau(C, C) = \\omega_1$, $\\tau(C, D) = \\omega_1$, $\\tau(D, C) = \\omega_1$, $\\tau(D, D) = \\omega_1$: neither agent has any influence in this environment.<\/li>\n  <li>$\\tau(C, C) = \\omega_1$, $\\tau(C, D) = \\omega_1$, $\\tau(D, C) = \\omega_2$, $\\tau(D, D) = \\omega_2$: the environment is sensitive to actions of only one agent.<\/li>\n<\/ul>\n\n<h2 id=\"rational-action\">Rational Action<\/h2>\n\n<p>Suppose we have the case where both agents can influence the outcome, and they have utility functions as follows:<\/p>\n\n\\[U_i(C, C) = 4, \\quad U_i(C, D) = 4, \\quad U_i(D, C) = 1, \\quad U_i(D, D) = 1\\]\n\n\\[U_j(C, C) = 4, \\quad U_j(C, D) = 1, \\quad U_j(D, C) = 4, \\quad U_j(D, D) = 1\\]\n\n<p>The preferences of agent i are:<\/p>\n\n\\[\\langle C, C \\rangle \\succeq_i \\langle C, D \\rangle \\succ_i \\langle D, C \\rangle \\succeq_i \\langle D, D \\rangle\\]\n\n<p>Cooperate is a better choice for $i$, and $i$ will choose cooperate.<\/p>\n\n<p>We can characterize the previous scenario in a <strong>payoff matrix<\/strong>:<\/p>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/payoff-matrix.png\" alt=\"Payoff Matrix\" \/>\n  \n  <figcaption class=\"caption-text\">payoff matrix<\/figcaption>\n  \n<\/figure>\n\n<p>or in a more simplified form:<\/p>\n\n\\[\\left(\n        \\begin{matrix}\n            (U_i(D, D), U_j(D, D)) &amp; (U_i(C, D), U_j(C, D)) \\\\\n            (U_i(D, C), U_j(D, C)) &amp; (U_i(C, C), U_j(C, C))\n        \\end{matrix}\n    \\right)\n    =\n    \\left(\n        \\begin{matrix}\n            (1, 1) &amp; (4, 1) \\\\\n            (1, 4) &amp; (4, 4)\n        \\end{matrix}\n    \\right)\\]\n\n<p>where agent $i$ is the column player, and agent $j$ is the row player.<\/p>\n\n<h2 id=\"zero-sum-game-and-nonzero-sum-game\">Zero-Sum Game and Nonzero-Sum Game<\/h2>\n\n<p>A game is <strong>zero-sum<\/strong> if the sum of utilities across all agents for every possible outcome is zero:<\/p>\n\n\\[\\forall (ac_i, ac_j) \\in Ac \\times Ac \\rightarrow U_i(ac_i, ac_j) + U_j(ac_i, ac_j) = 0\\]\n\n<p>or more broadly defined as:<\/p>\n\n\\[\\forall \\omega \\in \\Omega \\rightarrow \\sum_i U_i(\\omega) = 0\\]\n\n<p>An example payoff matrix for a zero-sum game:<\/p>\n\n\\[\\left(\n        \\begin{matrix}\n            (1, -1) &amp; (-3, 3) \\\\\n            (0, 0) &amp; (-4, 4)\n        \\end{matrix}\n    \\right)\\]\n\n<p>In this <strong>strict competition<\/strong> scenario, pie is fixed. One player win necessarily leads to the other player losing, and vice versa.<\/p>\n\n<p>A game is <strong>non-zero-sum<\/strong> if there exists at least one outcome where utilities do not sum to zero:<\/p>\n\n\\[\\exists (ac_i, ac_j) \\in Ac \\times Ac \\rightarrow U_i(ac_i, ac_j) + U_j(ac_i, ac_j) \\neq 0\\]\n\n<p>or more broadly defined as:<\/p>\n\n\\[\\exists \\omega \\in \\Omega \\rightarrow \\sum_i U_i(\\omega) \\neq 0\\]\n\n<p>This scenario may allow cooperative outcomes (variable-pie). It is a more general case in the real world. The interesting thing is that <strong>zero-sum<\/strong> encounters in real life are very rare, but people tend to act in many scenarios as if they were zero-sum.<\/p>\n\n<hr \/>\n\n<h1 id=\"solution-concepts\">Solution Concepts<\/h1>\n\n<p>A rational agent behave in a <strong>rational manner<\/strong> in the given scenario according to some strategy derived from previous research. In this section, we will focus on some of them.<\/p>\n\n<h2 id=\"dominant-strategies\">Dominant Strategies<\/h2>\n\n<p>Given any particular strategy (either $C$ or $D$) of agent $i$, there will be a number of possible outcomes. We say $s_1$ dominates $s_2$ for $i$ if every outcome possible by $i$ playing $s_1$ is preferred over every outcome possible by $i$ playing $s_2$. A rational agent will never play a dominated strategy.<\/p>\n\n\\[\\left(\n        \\begin{matrix}\n            (1, 1) &amp; (4, 1) \\\\\n            (1, 4) &amp; (4, 4)\n        \\end{matrix}\n    \\right)\\]\n\n<p>In previous example, $C$ dominates $D$ for both players. So in deciding what to do, we can <strong>delete dominated strategies<\/strong>. Unfortunately, there isn\u2019t always a unique undominated strategy. However, dominant strategy equilibrium can be obtained by each player chooses her dominant strategy.<\/p>\n\n<h2 id=\"nash-equilibrium-strategies\">Nash Equilibrium Strategies<\/h2>\n\n<p>In general, we will say that two strategies $s_1$ (for $i$) and $s_2$ (for $j$) are in <strong>Nash equilibrium<\/strong> if:<\/p>\n<ul>\n  <li>under the assumption that agent $i$ plays $s_1$, agent $j$ can do no better than play $s_2$<\/li>\n  <li>under the assumption that agent $j$ plays $s_2$, agent $i$ can do no better than play $s_1$<\/li>\n<\/ul>\n\n<p>More mathematically, we say that $s_1$ for agent $i$ and $s_2$ for agent $j$ are in Nash equilibrium if:<\/p>\n\n\\[\\neg \\exists s_1', s_2' \\in Ac \\times Ac \\text{ s.t. } U_i(s_1', s_2) &gt; U_i(s_1, s_2) \\text{ or } U_j(s_1, s_2') &gt; U_j(s_1, s_2)\\]\n\n<p>Unfortunately, the fact is that:<\/p>\n<ul>\n  <li><strong>Not<\/strong> every interaction scenario has a <strong>pure strategy<\/strong> Nash equilibrium.<\/li>\n  <li>Some interaction scenarios have <strong>more<\/strong> than one <strong>pure strategy<\/strong> Nash equilibrium.<\/li>\n<\/ul>\n\n<p>Note that <strong>pure strategy<\/strong> here means that each player only choose one action to play. In opposite concept, if each player can choose an action according to a probability <strong>distribution<\/strong>, then we call it <strong>mixed strategy<\/strong>. Let\u2019s see an example:<\/p>\n\n<p>Players $i$ and $j$ simultaneously choose the face of a coin, either \u201cheads\u201d or \u201ctails\u201d. If they show the same face, then $i$ wins, while if they show different faces, then $j$ wins. By the way, this is the <strong>zero-sum<\/strong> game. And the payoff matrix can be set as:<\/p>\n\n\\[\\left(\n        \\begin{matrix}\n            (-1, 1) &amp; (1, -1) \\\\\n            (1, -1) &amp; (-1, 1)\n        \\end{matrix}\n    \\right)\\]\n\n<p>Can tell no pair of strategies forms a <strong>pure strategy<\/strong> Nash equilibrium: whatever pair of strategies is chosen, somebody will wish they had done something else. In other words, there is always motivated to change their strategy. To solve this problem, we can play the action under distribution.<\/p>\n\n<p>Let\u2019s first extend <strong>pure strategy<\/strong> Nash equilibrium to <strong>mixed strategy<\/strong> Nash equilibrium. The utility of mixed strategy can be expressed as:<\/p>\n\n\\[U_i(\\sigma_i, \\sigma_j) = \\sum_{a_i \\in Ac_i} \\sum_{a_j \\in Ac_j} \\sigma_i(a_i) \\sigma_j(a_j) \\cdot U_i(a_i, a_j).\\]\n\n<p>where $\\Sigma_i$ is player $i$\u2019s mix strategy distribution space, $\\sigma_i$ is player $i$\u2019s one mix strategy. Mix strategy pairs $(\\sigma_i, \\sigma_j)$ are in <strong>Nash equilibrium<\/strong> if:<\/p>\n\n\\[\\neg \\exists \\sigma_i', \\sigma_j' \\in \\Sigma_i \\times \\Sigma_j \\text{ s.t. } U_i(\\sigma_i', \\sigma_j) &gt; U_i(\\sigma_i, \\sigma_j) \\text{ or } U_j(\\sigma_i, \\sigma_j') &gt; U_j(\\sigma_i, \\sigma_j)\\]\n\n<p>Mixed strategies to play \u201cheads\u201d with probability $0.5$ and play \u201ctails\u201d with probability $0.5$ for both players are in Nash equilibrium. This is an example of a <strong>mixed strategy<\/strong> Nash equilibrium.<\/p>\n\n<p>It can be proved that <strong>every finite game has a Nash equilibrium in mixed strategies<\/strong> based on <strong>fixed point theorem (Brouwer\/Kakutani)<\/strong>.<\/p>\n\n<h2 id=\"pareto-optimality-strategies\">Pareto Optimality Strategies<\/h2>\n\n<p>An outcome is said to be Pareto optimal (or Pareto efficient) if there is no other outcome that makes one agent better off without making another agent worse off.<\/p>\n\n<p>Formally, $\\omega$ is said to be Pareto optimal (or Pareto efficient) if there is no other outcome $\\omega\u2019$ such that:<\/p>\n\n\\[\\begin{align*}\n                &amp; U_i(\\omega) \\leq U_i(\\omega') \\\\\n   \\text{AND }  &amp; U_j(\\omega) \\leq U_j(\\omega') \\\\\n   \\text{AND }  &amp; [U_i(\\omega) &lt; U_i(\\omega') \\text{ OR } U_j(\\omega) &lt; U_j(\\omega')]\n\\end{align*}\\]\n\n<p>If an outcome is Pareto optimal, then at least one agent will be reluctant to move away from it (because this agent will be worse off). If an outcome $\\omega$ is not Pareto optimal, then there is another outcome $\\omega\u2019$ that makes everyone as happy, if not happier, than $\\omega$.<\/p>\n\n<p>In the <strong>economic<\/strong> sense:<\/p>\n<ul>\n  <li>Nash equilibrium is the inevitable result of <strong>individual rationality<\/strong>, but it may fall into the trap of \u201ccollective irrationality\u201d, which require <strong>external intervention<\/strong> (such as taxation and agreements).<\/li>\n  <li>Pareto optimality is the ideal state of <strong>collective rationality<\/strong>, but it may require forced cooperation or institutional design to achieve it. Through rule design (such as auction rules), <strong>Nash equilibrium<\/strong> can be <strong>brought close<\/strong> to <strong>Pareto optimality<\/strong>.<\/li>\n  <li>The inequality between the two is the core logic of game theory to explain social contradictions (such as the tragedy of the commons and price wars).<\/li>\n<\/ul>\n\n<h2 id=\"max-social-welfare-strategies\">Max Social Welfare Strategies<\/h2>\n\n<p>The social welfare (think of it as the \u201c<strong>total amount of wealth<\/strong> in the system\u201d) of an outcome $\\omega$ is the sum of the utilities that each agent gets from $\\omega$:<\/p>\n\n\\[\\sum_{i \\in Ag} U_i(\\omega)\\]\n\n<p>As a solution concept, may be appropriate when the whole system (all agents) has a single owner (then overall benefit of the system is important, not individuals).<\/p>\n\n<hr \/>\n\n<h1 id=\"the-prisoners-dilemma\">The Prisoner\u2019s Dilemma<\/h1>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/prisoner-dilemma.png\" alt=\"The Prisoner's Dilemma\" \/>\n  \n  <figcaption class=\"caption-text\">the prisoner's dilemma<\/figcaption>\n  \n<\/figure>\n\n<p>We can assume the following payoff matrix to describe the Prisoner\u2019s Dilemma:<\/p>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/prisoner-dilemma-payoff-matrix.png\" alt=\"Prisoner's Dilemma Payoff Matrix\" \/>\n  \n  <figcaption class=\"caption-text\">prisoner's dilemma payoff matrix<\/figcaption>\n  \n<\/figure>\n\n<ul>\n  <li>Top left: If both defect, then both get punishment for mutual defection.<\/li>\n  <li>Top right: If $i$ cooperates and $j$ defects, $i$ gets sucker\u2019s payoff of $1$, while $j$ gets $4$.<\/li>\n  <li>Bottom left: If $j$ cooperates and $i$ defects, $j$ gets sucker\u2019s payoff of $1$, while $i$ gets $4$.<\/li>\n  <li>Bottom right: Reward for mutual cooperation.<\/li>\n<\/ul>\n\n<p>The individual rational action is defect which guarantees a payoff of no worse than $2$, whereas cooperating guarantees a payoff of at most $1$. So defection is the best response to all possible strategies: both agents defect, and get payoff $= 2$.<\/p>\n\n<p>But intuition says this is not the best outcome. Surely they should both cooperate and each get payoff of $3$!<\/p>\n<ul>\n  <li>$D$ is a dominant strategy<\/li>\n  <li>$\\langle D,D \\rangle$ is the only Nash equilibrium.<\/li>\n  <li>All outcomes except $\\langle D,D \\rangle$ are Pareto optimal.<\/li>\n  <li>$\\langle C,C \\rangle$ maximises social welfare.<\/li>\n<\/ul>\n\n<p>This apparent paradox is the fundamental problem of multi-agent interactions. It appears to imply that cooperation will not occur in societies of self-interested agents. An example in the real world is the <strong>nuclear arms reduction<\/strong>. The core difficulty here is how to recover cooperation.<\/p>\n\n<p>The strategy we really want to play in the prisoner\u2019s dilemma is that <strong>I\u2019ll cooperate if he will<\/strong>. Program equilibria provide one way of enabling this. Each agent submits a program strategy to a mediator which jointly executes the strategies. Crucially, strategies can be conditioned on the strategies of the others.<\/p>\n\n<p>In <strong>The Iterated Prisoner\u2019s Dilemma<\/strong>, If we know we will be meeting our opponent again, then the incentive to defect appears to evaporate. Cooperation is the rational choice in the <strong>infinitely repeated<\/strong> prisoner\u2019s dilemma. However, playing the prisoner\u2019s dilemma with a fixed, <strong>finite<\/strong>, <strong>pre-determined<\/strong>, commonly known number of rounds, <strong>defection<\/strong> is the best strategy. This is the backwards induction problem.<\/p>\n\n<p><strong>Suppose we play iterated prisoner\u2019s dilemma against a range of opponents, what strategy should we choose, so as to maximize our overall payoff?<\/strong> Axelrod (1984) investigated this problem with a computer tournament for programs playing the prisoner\u2019s dilemma. His research suggests the following rules for succeeding:<\/p>\n<ul>\n  <li><strong>Don\u2019t be envious<\/strong>: Don\u2019t play as if it were zero sum!<\/li>\n  <li><strong>Be nice<\/strong>: Start by cooperating, and reciprocate cooperation.<\/li>\n  <li><strong>Retaliate appropriately<\/strong>: Always punish defection immediately, but use \u201cmeasured\u201d force.<\/li>\n  <li><strong>Don\u2019t hold grudges<\/strong>: Always reciprocate cooperation immediately!<\/li>\n<\/ul>\n\n<p>By the way this interesting problem is set to be the assignment, which quiet surprises me.<\/p>\n\n<hr \/>\n\n<p>Related Posts \/ Websites \ud83d\udc47<\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Intelligent-Agent-Prolegomenon\">Ray - NTU SC4003 Lecture 3 Note: Intelligent Agent Prolegomenon<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Agent-Decision-Making\">Ray - NTU SC4003 Lecture 4 Note: Agent Decision Making<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Agent-Architecture\">Ray - NTU SC4003 Lecture 5 Note: Agent Architecture<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Working-Together-Benevolent-Cooperative-Agents\">Ray - NTU SC4003 Lecture 7 Note: Working Together Benevolent\/Cooperative Agents<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"Allocating-Scarce-Resources-Auction#vickrey-auctions\">Ray - NTU SC4003 Lecture 9 Note: Allocating Scarce Resources: Auction<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Making-Group-Decisions-Voting\">Ray - NTU SC4003 Lecture 10 Note: Voting Mechanism<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Forming-Coalition\">Ray - NTU SC4003 Lecture 11 Note: Forming Coalition<\/a><\/p>\n","pubDate":"Mon, 24 Mar 2025 00:00:00 +0000","link":"https:\/\/huruilizhen.github.io\/Self-Interested-Agents-Game-Theory-Foundation","guid":"https:\/\/huruilizhen.github.io\/Self-Interested-Agents-Game-Theory-Foundation","category":["intelligent-agent","game-theory"]},{"title":"Working Together Benevolent\/Cooperative Agents","description":"<p>In the previous related posts, we discussed how agents can make decisions and act in an environment. However, we often have <strong>multiple agents<\/strong> working together in a <strong>multi-agent system<\/strong>. In this chapter, we will discuss why and how agents work together. Since agents are autonomous, they have to make decisions at run-time, and be capable of dynamic coordination. This coordination requires agents to <strong>share tasks and information<\/strong>. But, if agents are designed by different individuals, they may not have common goals. Therefore, we need to make a <strong>distinction<\/strong> between <strong>benevolent<\/strong> agents and <strong>self-interested<\/strong> agents. In this chapter, we will focus on benevolent agents, which are agents that are designed to <strong>work together<\/strong> and <strong>achieve a common goal<\/strong>. This post discuss cooperative agents in Lecture 7 of the <code class=\"language-plaintext highlighter-rouge\">SC4003<\/code> course in NTU.<\/p>\n\n<!--more-->\n\n<p><strong>Benevolent agents<\/strong> are designed to help each other whenever asked, as <strong>our best interest is their best interest<\/strong>. This simplifies the system design task enormously, and in this post we first focus on cooperative distributed problem solving (CDPS).<\/p>\n\n<p>However, in more general cases, agents may represent the <strong>interests of individuals or organisations<\/strong>, which cannot be assumed to be benevolent. Instead, agents will act to further their own interests, possibly at the expense of others, which may lead to conflicts (the so called <strong>self-interested agents<\/strong>). This greatly complicates the design task, and may require strategic behavior, such as reaching agreements and using game theory, which will be covered in the later posts.<\/p>\n\n<p>There are basically two Criteria for assessing an agent-based system, <strong>coherence<\/strong> and <strong>cooperation<\/strong>. Coherence means that how well the multiagent system <strong>behaves as a unit<\/strong> along some dimension of evaluation , while Cooperation means the  the degree to which the agents can <strong>avoid extraneous activity<\/strong> such as Synchronizing and aligning their activities.<\/p>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/work-together.png\" alt=\"Work Together\" \/>\n  \n  <figcaption class=\"caption-text\">Hybrid Architectures<\/figcaption>\n  \n<\/figure>\n\n<p>In the context of <strong>Cooperative distributed problem solving (CDPS)<\/strong>, we mainly focus on the following three stages:<\/p>\n<ul>\n  <li><strong>Problem decomposition<\/strong>: The overall problem to be solved is divided into smaller sub-problems. Typically a <strong>recursive\/hierarchical process<\/strong>, where subproblems get divided up also. The process will focus on how this is done and who does the division.<\/li>\n  <li><strong>Sub-problem solving<\/strong>: The sub-problems derived in the previous stage are solved. Agents typically <strong>share<\/strong> some <strong>information<\/strong> during this process. A given step may involve two agents <strong>synchronizing<\/strong> their <strong>actions<\/strong>.<\/li>\n  <li><strong>Answer synthesis<\/strong>: In this stage solutions to sub-problems are <strong>integrated<\/strong>. This may be <strong>hierarchical<\/strong>. Solutions in different abstract levels will be different.<\/li>\n<\/ul>\n\n<h2 id=\"table-of-contents\">Table of Contents<\/h2>\n<ul>\n  <li><a href=\"#task-sharing---the-contract-net\">Task Sharing - The Contract Net<\/a>\n    <ul>\n      <li><a href=\"#stages-of-contract-net\">Stages of Contract Net<\/a><\/li>\n      <li><a href=\"#issues-for-implementing-contract-net\">Issues for Implementing Contract Net<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#result-sharing\">Result Sharing<\/a>\n    <ul>\n      <li><a href=\"#result-sharing-in-blackboard-systems\">Result Sharing in Blackboard Systems<\/a><\/li>\n      <li><a href=\"#result-sharing-in-subscribenotify-pattern\">Result Sharing in Subscribe\/Notify Pattern<\/a><\/li>\n    <\/ul>\n  <\/li>\n  <li><a href=\"#handling-inconsistency\">Handling Inconsistency<\/a><\/li>\n  <li><a href=\"#coordination\">Coordination<\/a>\n    <ul>\n      <li><a href=\"#social-norms\">Social Norms<\/a><\/li>\n      <li><a href=\"#joint-intentions\">Joint Intentions<\/a><\/li>\n    <\/ul>\n  <\/li>\n<\/ul>\n\n<hr \/>\n\n<h1 id=\"task-sharing---the-contract-net\">Task Sharing - The Contract Net<\/h1>\n\n<p>Given this model of cooperative problem solving, we have two activities that are likely to be present:<\/p>\n<ul>\n  <li><strong>task sharing<\/strong>: components of a task are distributed to component agents; how do we decide how to allocate tasks to agents?<\/li>\n  <li><strong>result sharing<\/strong>: information (partial results etc) is distributed; how do we assemble a complete solution from the parts?<\/li>\n<\/ul>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/task-result-sharing.png\" alt=\"Task and Result Sharing\" \/>\n  \n  <figcaption class=\"caption-text\">task and result sharing<\/figcaption>\n  \n<\/figure>\n\n<h2 id=\"stages-of-contract-net\">Stages of Contract Net<\/h2>\n\n<p>Well known <strong>task-sharing protocol<\/strong> for <strong>task allocation<\/strong> is the <strong>contract net<\/strong>, which includes five stages, Recognition, Announcement, Bidding, Awarding, and Expediting. Let\u2019s have a look at each stage.<\/p>\n\n<p>In stage of <strong>recognition<\/strong>, an agent recognises it has a problem it wants help with. Agent has a goal, and either realises it cannot achieve the goal in isolation since it <strong>does not have capability<\/strong>, or realises it would <strong>prefer not<\/strong> to achieve the goal <strong>in isolation<\/strong> (typically because of solution quality, deadline, etc). As a result, it needs to involve other agents.<\/p>\n\n<p>During the <strong>announcement<\/strong> stage, the agent with the task sends out an announcement of the task which includes a <strong>specification of the task<\/strong> to be achieved. Specification must encode <strong>description<\/strong> of task itself, any <strong>constraints<\/strong> (e.g., deadlines, quality constraints), and <strong>meta-task information<\/strong> (e.g., \u201cbids must be submitted by\u2026\u201d). The announcement is then broadcast.<\/p>\n\n<p>Agents that receive the announcement decide for themselves whether they wish to <strong>bid<\/strong> for the task. The key factors are that they must decide whether it is <strong>capable of expediting task<\/strong> and <strong>determine<\/strong> quality <strong>constraints<\/strong> and price information. If they do choose to bid, then they submit a <strong>tender<\/strong>.<\/p>\n\n<p>Agent that sent task announcement must choose between bids and decide who to <strong>award<\/strong> the contract to. The result of this process is communicated to agents that submitted a bid. The successful contractor then <strong>expedites<\/strong> the task. In some case, the process may involve generating further manager-contractor relationships: sub-contracting.<\/p>\n\n<figure class=\"image-card width-wide caption\">\n  <img src=\"\/images\/task-sharing-stages.png\" alt=\"Task Sharing Stages\" \/>\n  \n  <figcaption class=\"caption-text\">task sharing stages<\/figcaption>\n  \n<\/figure>\n\n<p>The collection of nodes is the <strong>contract net<\/strong>. Each node on the network can, at different times or for different tasks, be a manager or a contractor. When a node <strong>gets a composite task<\/strong> (or for any reason can\u2019t solve its present task), it breaks it into <strong>subtasks<\/strong> (if possible) and <strong>announces<\/strong> them (acting as a manager), <strong>receives bids<\/strong> from potential contractors, then <strong>awards<\/strong> the job (example domain: network resource management, printers, ).<\/p>\n\n<h2 id=\"issues-for-implementing-contract-net\">Issues for Implementing Contract Net<\/h2>\n\n<p>To effectively manage task allocation and execution, it is crucial to address several key aspects: <strong>specifying tasks clearly<\/strong>, <strong>defining the quality of service required<\/strong>, <strong>deciding on bidding strategies<\/strong>, <strong>selecting between competing offers<\/strong>, and <strong>differentiating offers based on multiple criteria<\/strong>. These considerations ensure a structured and efficient approach to task management in a multi-agent environment.<\/p>\n\n<p>On the question how to bid. We say At <strong>time<\/strong> $t$ a <strong>contractor<\/strong> $i$ is scheduled to carry out <strong>combine task<\/strong> $T_i$. Contractor $i$ also has resources $e$. Then $i$ receives an announcement of task specification $ts$, which is for a set of tasks $T(ts)$. These will cost $i$ $c(T(ts))$ to carry out. The marginal cost of carrying out $T(ts)$ will be:<\/p>\n\n\\[u(T(ts) | T_i) = c(T_i \\cup T(ts)) \u2013 c(T_i)\\]\n\n<p>which is the difference between carrying out what it has already agreed to do and what it has already agreed plus the new tasks. Due to <strong>synergies<\/strong>, the marginal cost is often <strong>less<\/strong> than $c(T(ts))$. In fact, it can be zero \u2014 the additional tasks can be done for free. As long as $u(T(ts) | T_i) &lt; e$ then the agent can afford to do the new work, then it is rational for the agent to bid for the work, otherwise not.<\/p>\n\n<hr \/>\n\n<h1 id=\"result-sharing\">Result Sharing<\/h1>\n\n<p>In results sharing, agents <strong>provide<\/strong> each other with <strong>information<\/strong> as they work towards a solution. This collaboration leads to improved problem solving since independent pieces of a <strong>solution<\/strong> can be <strong>cross-checked<\/strong>, <strong>combining<\/strong> local <strong>views<\/strong> can achieve a better overall view, and shared results can improve the <strong>accuracy<\/strong> of results. Additionally, sharing results allows the use of <strong>parallel resources<\/strong> on a problem, which can <strong>accelerate<\/strong> the <strong>derivation<\/strong> of a solution.<\/p>\n\n<h2 id=\"result-sharing-in-blackboard-systems\">Result Sharing in Blackboard Systems<\/h2>\n\n<p>A group of specialists are seated in a room with a large blackboard. They work as a team to brainstorm a solution to a problem, <strong>using the blackboard as the workplace<\/strong> for cooperatively developing the solution. The session begins when the problem specifications are written onto the blackboard. The specialists all watch the blackboard, <strong>looking for an opportunity to apply their expertise to the developing solution<\/strong>. When someone writes something on the blackboard that allows another specialist to apply their expertise, the second specialist records their contribution on the blackboard, hopefully enabling other specialists to then apply their expertise. This process of adding contributions to the blackboard continues until the problem has been solved.<\/p>\n\n<p>The blackboard system represents the pioneering scheme for <strong>cooperative problem solving<\/strong>. It uses a shared data structure (BB) where <strong>multiple agents<\/strong> (KSs\/KAs) can <strong>read and write<\/strong>. Agents contribute by writing <strong>partial solutions<\/strong> to the BB, which can be organized in a <strong>hierarchical<\/strong> manner. However, <strong>mutual exclusion<\/strong> is crucial over the BB to prevent conflicts, which can become a bottleneck due to the need for a lock mechanism, resulting in non-concurrent activity.<\/p>\n\n<table>\n  <thead>\n    <tr>\n      <th>\u00a0<\/th>\n      <th>Knowledge Source (KS)<\/th>\n      <th>Knowledge Agent (KA)<\/th>\n    <\/tr>\n  <\/thead>\n  <tbody>\n    <tr>\n      <td>Emphasis<\/td>\n      <td>Knowledge module<\/td>\n      <td>Active agent<\/td>\n    <\/tr>\n    <tr>\n      <td>Trigger<\/td>\n      <td>Passive, triggered by blackboard<\/td>\n      <td>Active, perceives and acts on new information<\/td>\n    <\/tr>\n    <tr>\n      <td>Autonomy<\/td>\n      <td>Low, knowledge is applied when triggered<\/td>\n      <td>High, more autonomous in perceiving and acting<\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n\n<h2 id=\"result-sharing-in-subscribenotify-pattern\">Result Sharing in Subscribe\/Notify Pattern<\/h2>\n\n<p>Subscribe\/Notify is a <strong>design pattern<\/strong> where objects register interest in specific events and are <strong>proactively notified when those events occur<\/strong>, enabling efficient and timely result sharing in dynamic systems. Rather than <strong>continuously polling<\/strong> other objects to check for changes, subscribers simply declare their interests once, and then <strong>rely on the system<\/strong> to alert them when <strong>relevant updates arise<\/strong>. This shift from pull-based to push-based communication significantly reduces unnecessary computational overhead and improves responsiveness, making it especially suitable for distributed, event-driven environments.<\/p>\n\n<p>In this pattern, objects must maintain awareness of each other\u2019s interests, typically by <strong>managing a subscription list<\/strong> that records which objects should be notified for each type of event. When a triggering event occurs, the publisher automatically notifies all relevant subscribers, allowing each subscriber to immediately act upon the new information without delay. As a result, information is proactively shared among objects, <strong>promoting real-time collaboration<\/strong> and more cohesive system behavior.<\/p>\n\n<p>Within the context of cooperative agents and result sharing, the Subscribe\/Notify pattern provides an elegant mechanism for <strong>synchronizing distributed knowledge contributions.<\/strong> Agents can specialize in different aspects of a problem, subscribe to updates relevant to their expertise, and progressively build a shared understanding without requiring centralized control or direct, constant querying. This fosters scalability, reduces communication complexity, and enhances the ability of multi-agent systems to solve complex problems collaboratively.<\/p>\n\n<hr \/>\n\n<h1 id=\"handling-inconsistency\">Handling Inconsistency<\/h1>\n\n<p>A group of agents may have <strong>inconsistencies<\/strong> in their <strong>beliefs<\/strong> or <strong>goals\/intentions<\/strong>. Inconsistent beliefs arise because agents have different <strong>views of the world<\/strong>, which may be due to <strong>sensor faults<\/strong> or <strong>noise<\/strong> or just because they can\u2019t see everything. Inconsistent goals may arise because agents are built by different people with different objectives. There are three classical ways to handle inconsistencies:<\/p>\n<ul>\n  <li><strong>Do not allow it<\/strong>: For example, in the contract net the only view that matters is that of the manager agent.<\/li>\n  <li><strong>Resolve inconsistency<\/strong>: Agents discuss the inconsistent information\/goals until the inconsistency goes away (argumentation).<\/li>\n  <li><strong>Build systems that degrade gracefully<\/strong> in the face of inconsistency FA\/C (Functionally accurate\/cooperative).<\/li>\n<\/ul>\n\n<p>In functionally accurate\/cooperative (FA\/C) systems, agents are specifically designed to handle incomplete, inconsistent, and outdated knowledge without requiring full synchronization. Instead of assuming that local knowledge bases must be complete and consistent at all times, agents solve problems <strong>opportunistically<\/strong> and <strong>incrementally<\/strong>, advancing solutions whenever possible <strong>based on partial information<\/strong>. This approach reduces computational overhead and makes systems far more robust in dynamic, uncertain environments.<\/p>\n\n<p>Rather than exchanging raw data, agents in FA\/C systems <strong>share high-level intermediate results<\/strong>, allowing them to collaboratively refine their understanding and correct inconsistencies implicitly through comparison and integration. <strong>Problem solving is not bound to a rigid sequence of steps<\/strong>; multiple alternative paths to a solution are allowed. If one path fails, agents can easily shift to another, ensuring that progress toward the goal continues. This decentralized, resilient problem-solving strategy enables cooperative agents to operate effectively <strong>even when global consistency is unattainable<\/strong>.<\/p>\n\n<hr \/>\n\n<h1 id=\"coordination\">Coordination<\/h1>\n\n<p>Coordination is managing dependencies between agents. There are some examples in the real world:<\/p>\n<ul>\n  <li>We both want to leave the room through the same door. we are walking such that we will arrive at the door at the same time. What do we do to ensure we can both get through the door?<\/li>\n  <li>We both arrive at the copy room with a stack of paper to photocopy. Who gets to use the machine first?<\/li>\n<\/ul>\n\n<h2 id=\"social-norms\">Social Norms<\/h2>\n\n<p>The human <strong>society<\/strong> are often <strong>regulated<\/strong> by (often unwritten) <strong>rules<\/strong> of behavior. E.g. the rule when boads bus at stops, first come first board. In an agent system, we can design the <strong>norms<\/strong> and program agents to follow them, or let norms evolve.<\/p>\n\n<p>Agents can be described as a function which, given a run ending in a state, gives us an action:<\/p>\n\n\\[Ag : R_E \\rightarrow Ac\\]\n\n<p>where $R_E$ is the set of all possible runs of agent $Ag$ in ending in a state $e \\in E$, $Ac$ is the action space.<\/p>\n\n<p>A constraint is then a pair:<\/p>\n\n\\[\\langle E', \\alpha \\rangle\\]\n\n<p>where $E\u2019 \\subseteq E$ and $\\alpha \\in Ac$. This constraint says that $\\alpha$ cannot be done in any state in $E\u2019$. A social law is a set of constraints<\/p>\n\n<h2 id=\"joint-intentions\">Joint Intentions<\/h2>\n\n<p>Joint intentions extend the idea of individual intentions to teams of agents, where agents form a collective commitment to achieving a goal (G), motivated by a shared reason (M). Levesque formalized this through the concept of a Joint Persistent Goal (JPG), where agents collectively maintain G until a termination condition is mutually believed.<\/p>\n\n<p>Initially, agents believe that G is not yet achieved but remains possible. They continue working towards G until it becomes mutually believed that (1) G is <strong>satisfied<\/strong>, (2) G is <strong>impossible<\/strong>, or (3) the <strong>motivation<\/strong> M is <strong>no longer valid<\/strong>. If an agent individually discovers that a termination condition is met, it does not immediately abandon G. Instead, <strong>it adopts a new goal: to ensure that this knowledge becomes mutually believed among all agents through communicatio<\/strong>n.<\/p>\n\n<p>This process guarantees coordinated team behavior, ensuring agents only stop pursuing G when everyone is fully informed. Mutual belief via communication is essential to synchronizing agent actions and properly concluding joint activities.<\/p>\n\n<blockquote>\n  <p>Joint intentions require agents to persist toward a goal until mutual belief of goal achievement, impossibility, or demotivation is established, ensuring coordinated termination through communication.<\/p>\n<\/blockquote>\n\n<hr \/>\n\n<p>Related Posts \/ Websites \ud83d\udc47<\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Intelligent-Agent-Prolegomenon\">Ray - NTU SC4003 Lecture 3 Note: Intelligent Agent Prolegomenon<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Agent-Decision-Making\">Ray - NTU SC4003 Lecture 4 Note: Agent Decision Making<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Agent-Architecture\">Ray - NTU SC4003 Lecture 5 Note: Agent Architecture<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Self-Interested-Agents-Game-Theory-Foundation\">Ray - NTU SC4003 Lecture 8 Note: Game Theory Foundations<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"Allocating-Scarce-Resources-Auction#vickrey-auctions\">Ray - NTU SC4003 Lecture 9 Note: Allocating Scarce Resources: Auction<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Making-Group-Decisions-Voting\">Ray - NTU SC4003 Lecture 10 Note: Voting Mechanism<\/a><\/p>\n\n<p>\ud83d\udcd1 <a href=\"\/Forming-Coalition\">Ray - NTU SC4003 Lecture 11 Note: Forming Coalition<\/a><\/p>\n","pubDate":"Sun, 16 Mar 2025 00:00:00 +0000","link":"https:\/\/huruilizhen.github.io\/Working-Together-Benevolent-Cooperative-Agents","guid":"https:\/\/huruilizhen.github.io\/Working-Together-Benevolent-Cooperative-Agents","category":"intelligent-agent"}]}}