libgit2 · petersalomonsen · Mar 1, 2019
diff --git a/.gitignore b/.gitignore
@@ -31,6 +31,8 @@ msvc/Release/
 .*.swp
 tags
 mkmf.log
+
+.vscode
 *.profdata
 *.profraw
 CMakeSettings.json

diff --git a/README.md b/README.md
@@ -366,6 +366,10 @@ Here are the bindings to libgit2 that are currently available:
 If you start another language binding to libgit2, please let us know so
 we can add it to the list.
 
+Experimental Emscripten support for running in a web-browser
+============================================================
+See [EMSCRIPTEN_HACKS](emscripten_hacks/README.md)
+
 How Can I Contribute?
 ==================================
 

diff --git a/azure-pipelines.yml b/azure-pipelines.yml
@@ -131,6 +131,16 @@ jobs:
         CMAKE_OPTIONS: -DDEPRECATE_HARD=ON
         SKIP_SSH_TESTS: true
 
+- job: emscripten
+  displayName: 'Emscripten (javascript)'
+  pool:
+    vmImage: 'Ubuntu 16.04'
+  steps:
+  - script: |
+      docker run --rm -v $(pwd):/src trzeci/emscripten bash emscripten_hacks/build.sh
+      cd emscripten_hacks
+      node nodetest.js
+      node nodefstest.js
 - job: documentation
   displayName: 'Generate Documentation'
   pool:

diff --git a/emscripten_hacks/.gitignore b/emscripten_hacks/.gitignore
@@ -0,0 +1,10 @@
+src
+deps
+CMake*
+libgit2.pc
+Makefile
+node_modules
+libgit2.js
+libgit2.wasm
+libgit2_node.js
+libgit2_node.wasm
diff --git a/emscripten_hacks/README.md b/emscripten_hacks/README.md
@@ -0,0 +1,71 @@
+Build for webassembly using Emscripten
+======================================
+
+This is an example of exposing basic operations of libgit2 to javascript for use in the browser and nodejs. The library is compiled to webassembly using [emscripten](https://emscripten.org)
+
+# Build using Emscripten
+
+First of all you need to source the emscripten sdk from where emscripten is located on your system:
+
+    source ./emsdk_env.sh
+
+Then from this folder, run the shell script:
+
+    bash build.sh
+
+When complete you should see the files `libgit2.js` / `libgit2.wasm` and `libgit2_node.js` / `libgit2_node.wasm`.
+
+# Build using Docker
+
+If you're not able to install emscripten, there's a simple alternative using docker (which is also used in the Azure build pipeline).
+
+Simply from the repository root directory, run:
+
+`docker run --rm -v $(pwd):/src trzeci/emscripten bash emscripten_hacks/build.sh`
+
+# Use in the browser
+Because of CORS restrictions in the browser you cannot read from github directly from another domain. You need to add a proxy on your web server. You can run the githttpproxy.js script in this folder to 
+get a local webserver with proxy to github.com:
+
+    node githttpproxy.js
+
+Navigate your browser to `http://localhost:5000`
+
+When testing with the index.html file included here you should open the web console which will prompt "ready" when loaded. Remember to switch to the libgit2 webworker for typing commands in the console.
+
+Type the commands:
+
+    jsgitinit();
+    jsgitclone("https://github.com/pathto/mygitrepo.git","mygitrepo");
+
+You'll see the git clone process starts. This is not a small repository so it takes some time.
+
+When the clone is done you can list the folder contents by typing:
+
+    FS.readdir("mygitrepo")
+
+A simple demonstration video can be seen here: https://youtu.be/rcBluzpUWE4
+
+## Why is a webworker needed?
+
+http requests from libgit2 are synchronous, and operations such as clone / pull  / push may take some times which would hang if run on the browser main thread. WebWorkers support both synchronous http requests and also working excellent for long running operations.
+
+# Use in nodejs
+
+You can also use the library from nodejs. The node version is different since it uses the nodejs HTTP API. Also synchronous HTTP is not straight forward with node, so this is solved by running a separate process for HTTP and call this using `execSync`.
+
+```
+const lg = require('libgit2_node.js');
+lg.onRuntimeInitialized = () => {
+    const FS = lg.FS;
+    const MEMFS = FS.filesystems.MEMFS;
+
+    FS.mkdir('/working');
+    FS.mount(MEMFS, { root: '.' }, '/working');
+    FS.chdir('/working');
+
+    jsgitinit(); // Initialize the library
+    jsgitclone("https://somehost/path-to-gitrepository.git","destinationfolder"); // clone a repository
+    console.log(FS.readdir("destinationfolder")); // Read the cloned directory contents
+};
+```
diff --git a/emscripten_hacks/build.sh b/emscripten_hacks/build.sh
@@ -0,0 +1,18 @@
+if [[ "$PWD" == *emscripten_hacks ]]; then
+ echo 'current dir is emscripten_hacks'
+else
+ cd emscripten_hacks
+fi
+
+emcmake cmake -DCMAKE_C_FLAGS="-Oz" -DNO_MMAP=true -DREGEX_BACKEND=regcomp -DAPPLY_EMSCRIPTEN_HACKS=ON -DSONAME=OFF -DUSE_HTTPS=OFF -DBUILD_SHARED_LIBS=OFF -DTHREADSAFE=OFF -DBUILD_CLAR=OFF -DUSE_SSH=OFF ..
+emcmake cmake -build .
+emmake make
+
+echo "building libgit2.js (nodejs)"
+emcc -DEMSCRIPTEN_NODEJS=1 -s ALLOW_MEMORY_GROWTH=1 --post-js jsinit.js -s "EXTRA_EXPORTED_RUNTIME_METHODS=['FS']" -Oz jslib.c libgit2.a -Isrc -I../src -I../include -o libgit2_node.js
+
+echo "building libgit2.js (browser)"
+emcc -s ALLOW_MEMORY_GROWTH=1 --post-js jsinit.js -s "EXTRA_EXPORTED_RUNTIME_METHODS=['FS']" -Oz jslib.c libgit2.a -Isrc -I../src -I../include -o libgit2.js
+
+cp libgit2*.js npmdist/
+cp libgit2*.wasm npmdist/
diff --git a/emscripten_hacks/git_worker.js b/emscripten_hacks/git_worker.js
@@ -0,0 +1,10 @@
+importScripts("libgit2.js");
+Module['onRuntimeInitialized'] = () => {
+
+    const dir="workdir";
+    FS.mkdir(dir,"0777");
+    FS.mount(IDBFS, {}, '/'+dir);
+    FS.chdir("/"+dir);     
+
+    console.log('libgit2 ready in webworker.');
+};
diff --git a/emscripten_hacks/githttpproxy.js b/emscripten_hacks/githttpproxy.js
@@ -0,0 +1,49 @@
+/**
+ * Simple webserver with proxy to github.com
+ */
+
+const http = require('http');
+const https = require('https');
+const fs = require('fs');
+
+function onRequest(request, response) {
+    let path = request.url.substring(1);
+
+    if(path.indexOf('/') > -1) {  
+        const options = {
+            hostname: 'github.com',
+            port: 443,
+            path: request.url,
+            method: request.method
+        };
+
+        console.log(`Proxying ${options.method} request to ${options.hostname} with path ${options.path}`);
+        const proxy = https.request(options, function (res) {
+            res.pipe(response, {
+            end: true
+            });
+        });
+
+        request.pipe(proxy, {
+            end: true
+        });
+    } else {
+        if(path === '') {
+            path = 'index.html';
+        }
+
+        if(fs.existsSync(path)) {
+            if(path.indexOf('.js') === path.length-3) {
+                response.setHeader('Content-Type', 'application/javascript');
+            } else if(path.indexOf('.wasm') === path.length-3) {
+                response.setHeader('Content-Type', 'application/wasm');
+            }
+            response.end(fs.readFileSync(path));
+        } else {
+            response.statusCode = 404;
+            response.end('');
+        } 
+    }
+}
+
+http.createServer(onRequest).listen(5000);
diff --git a/emscripten_hacks/githttpserver.js b/emscripten_hacks/githttpserver.js
@@ -0,0 +1,45 @@
+/**
+ * This example will create a git http server to a repository on your local disk.
+ * Modify GIT_PROJECT_ROOT below to change the loction of your git repositories.
+ */
+
+const http = require('http');
+const path = require('path');
+const fs = require('fs');
+const cgi = require('cgi');
+
+const script = 'git';
+
+const gitcgi = cgi(script, {args: ['http-backend'],
+    stderr: process.stderr,
+    env: {
+        'GIT_PROJECT_ROOT': '/home/ubuntu/git',
+        'GIT_HTTP_EXPORT_ALL': '1',
+        'REMOTE_USER': '[email protected]' // Push requires authenticated users by default
+    }
+});
+
+http.createServer( (request, response) => {
+
+    let path = request.url.substring(1);
+
+    if(path === '') {
+        path = 'index.html';
+    }
+
+    console.log(request.url);
+    if (request.url.indexOf('/gitrepos/') === 0 ) {
+        request.url = request.url.substr('/gitrepos'.length);        
+        gitcgi(request, response);
+    } else if(fs.existsSync(path)) {
+        if(path.indexOf('.js') === path.length-3) {
+            response.setHeader('Content-Type', 'application/javascript');
+        } else if(path.indexOf('.wasm') === path.length-3) {
+            response.setHeader('Content-Type', 'application/wasm');
+        }
+        response.end(fs.readFileSync(path));
+    } else {
+        response.statusCode = 404;
+        response.end('');
+    }  
+}).listen(5000);
diff --git a/emscripten_hacks/index.html b/emscripten_hacks/index.html
@@ -0,0 +1,27 @@
+<html>
+    <head>
+
+    </head>
+    <body>
+        <style>
+            pre {
+                background-color: #ddd;
+                padding: 5px;
+            }
+        </style>
+        <p>libgit2 is started as a webworker. Open web console to test it (Right-click + inspect in Google Chrome).</p>
+
+        <p>In the console select git_worker.js to receive commands.</p>
+
+        <p>Start with typing the command: <pre>jsgitinit()</pre></p>
+
+        <p>Then try cloning using: <pre>jsgitclone("https://somehost/path-to-gitrepository.git","destinationfolder")</pre></p>
+
+        <p>Finally list the contents of the repository using <pre>FS.readdir("destinationfolder")</pre></p>
+
+    </body>
+    <script>
+        const gitworker = new Worker("git_worker.js");
+
+    </script>
+</html>