{"title":"Dima Kogan","link":[{"@attributes":{"href":"http:\/\/notes.secretsauce.netindex.xml","rel":"self"}},{"@attributes":{"href":"http:\/\/notes.secretsauce.net"}}],"updated":"2014-07-25T17:34:00Z","id":"http:\/\/notes.secretsauce.netindex.xml","entry":[{"title":"Simple gpx export from ridewithgps","author":{"name":"Dima Kogan"},"link":{"@attributes":{"href":"http:\/\/notes.secretsauce.net\/notes\/2026\/04\/04_simple-gpx-export-from-ridewithgps.html"}},"updated":"2026-04-04T17:21:00Z","published":"2026-04-04T17:21:00Z","id":"notes\/2026\/04\/04_simple-gpx-export-from-ridewithgps.html","category":[{"@attributes":{"scheme":"\/tags\/tools.html","term":"tools","label":"tools"}},{"@attributes":{"scheme":"\/tags\/data.html","term":"data","label":"data"}},{"@attributes":{"scheme":"\/tags\/gis.html","term":"GIS","label":"GIS"}},{"@attributes":{"scheme":"\/tags\/bike.html","term":"bike","label":"bike"}}],"content":"<p>\nThe <a href=\"https:\/\/tourdelospadres.weebly.com\/\">Tour de Los Padres<\/a> is coming! The race organizer post <a href=\"https:\/\/ridewithgps.com\/routes\/54493422\">the route on\nridewithgps<\/a>. This works, but has convoluted interfaces for people not wanting to\nuse their service. I just wrote a simple script to export their data into a\nplain .gpx file, <i>including<\/i> all the waypoints; their exporter omits those.\n<\/p>\n\n<p>\nI've seen two flavors of their data, so here're two flavors of the\n<code>gpx-from-ridewithgps.py<\/code> script:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-python\"><span style=\"color: #cdcd00;\">#<\/span><span style=\"color: #cdcd00;\">!\/usr\/bin\/python3<\/span>\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> sys\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> json\n\n<span style=\"color: #00cdcd; font-weight: bold;\">def<\/span> <span style=\"color: #0000ee; font-weight: bold;\">quote_xml<\/span>(s):\n    <span style=\"color: #00cdcd; font-weight: bold;\">return<\/span> s.replace(<span style=\"color: #00cd00;\">\"&amp;\"<\/span>, <span style=\"color: #00cd00;\">\"&amp;amp;\"<\/span>).replace(<span style=\"color: #00cd00;\">\"&lt;\"<\/span>, <span style=\"color: #00cd00;\">\"&amp;lt;\"<\/span>).replace(<span style=\"color: #00cd00;\">\"&gt;\"<\/span>, <span style=\"color: #00cd00;\">\"&amp;gt;\"<\/span>)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"Reading stdin\"<\/span>, <span style=\"color: #0000ee; font-weight: bold;\">file<\/span>=sys.stderr)\n<span style=\"color: #cdcd00;\">data<\/span> = json.load(sys.stdin)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(r<span style=\"color: #00cd00;\">\"\"\"&lt;?xml version=\"1.0\" encoding=\"UTF-8\"?&gt;<\/span>\n<span style=\"color: #00cd00;\">&lt;gpx version=\"1.1\" creator=\"gpx-from-ridewithgps.py\" xmlns=\"http:\/\/www.topografix.com\/GPX\/1\/1\"&gt;\"\"\"<\/span>)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> item <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> data[<span style=\"color: #00cd00;\">\"extras\"<\/span>]:\n    <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> item[<span style=\"color: #00cd00;\">\"type\"<\/span>] != <span style=\"color: #00cd00;\">\"point_of_interest\"<\/span>:\n        <span style=\"color: #00cdcd; font-weight: bold;\">continue<\/span>\n    <span style=\"color: #cdcd00;\">poi<\/span> = item[<span style=\"color: #00cd00;\">\"point_of_interest\"<\/span>]\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'  &lt;wpt lat=\"{poi[\"lat\"]}\" lon=\"{poi[\"lng\"]}\"&gt;'<\/span>)\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'    &lt;name&gt;{quote_xml(poi[\"name\"])}&lt;\/name&gt;'<\/span>)\n\n    <span style=\"color: #cdcd00;\">desc<\/span> = poi.get(<span style=\"color: #00cd00;\">\"description\"<\/span>,<span style=\"color: #00cd00;\">\"\"<\/span>)\n    <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> <span style=\"color: #0000ee; font-weight: bold;\">len<\/span>(desc):\n        <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'    &lt;desc&gt;{quote_xml(desc)}&lt;\/desc&gt;'<\/span>)\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'  &lt;\/wpt&gt;'<\/span>)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"  &lt;trk&gt;&lt;trkseg&gt;\"<\/span>)\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> pt <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> data.get(<span style=\"color: #00cd00;\">\"route\"<\/span>, {}).get(<span style=\"color: #00cd00;\">\"track_points\"<\/span>, []):\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'    &lt;trkpt lat=\"{pt[\"y\"]}\" lon=\"{pt[\"x\"]}\"&gt;&lt;ele&gt;{pt[\"e\"]}&lt;\/ele&gt;&lt;\/trkpt&gt;'<\/span>)\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"  &lt;\/trkseg&gt;&lt;\/trk&gt;\"<\/span>)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"&lt;\/gpx&gt;\"<\/span>)\n<\/pre>\n<\/div>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-python\"><span style=\"color: #cdcd00;\">#<\/span><span style=\"color: #cdcd00;\">!\/usr\/bin\/python3<\/span>\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> sys\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> json\n\n<span style=\"color: #00cdcd; font-weight: bold;\">def<\/span> <span style=\"color: #0000ee; font-weight: bold;\">quote_xml<\/span>(s):\n    <span style=\"color: #00cdcd; font-weight: bold;\">return<\/span> s.replace(<span style=\"color: #00cd00;\">\"&amp;\"<\/span>, <span style=\"color: #00cd00;\">\"&amp;amp;\"<\/span>).replace(<span style=\"color: #00cd00;\">\"&lt;\"<\/span>, <span style=\"color: #00cd00;\">\"&amp;lt;\"<\/span>).replace(<span style=\"color: #00cd00;\">\"&gt;\"<\/span>, <span style=\"color: #00cd00;\">\"&amp;gt;\"<\/span>)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"Reading stdin\"<\/span>, <span style=\"color: #0000ee; font-weight: bold;\">file<\/span>=sys.stderr)\n<span style=\"color: #cdcd00;\">data<\/span> = json.load(sys.stdin)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(r<span style=\"color: #00cd00;\">\"\"\"&lt;?xml version=\"1.0\" encoding=\"UTF-8\"?&gt;<\/span>\n<span style=\"color: #00cd00;\">&lt;gpx version=\"1.1\" creator=\"gpx-from-ridewithgps.py\" xmlns=\"http:\/\/www.topografix.com\/GPX\/1\/1\"&gt;\"\"\"<\/span>)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> poi <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> data[<span style=\"color: #00cd00;\">\"points_of_interest\"<\/span>]:\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'  &lt;wpt lat=\"{poi[\"lat\"]}\" lon=\"{poi[\"lng\"]}\"&gt;'<\/span>)\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'    &lt;name&gt;{quote_xml(poi[\"name\"])}&lt;\/name&gt;'<\/span>)\n\n    <span style=\"color: #cdcd00;\">desc<\/span> = poi.get(<span style=\"color: #00cd00;\">\"description\"<\/span>,<span style=\"color: #00cd00;\">\"\"<\/span>)\n    <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> <span style=\"color: #0000ee; font-weight: bold;\">len<\/span>(desc):\n        <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'    &lt;desc&gt;{quote_xml(desc)}&lt;\/desc&gt;'<\/span>)\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'  &lt;\/wpt&gt;'<\/span>)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> poi <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> data[<span style=\"color: #00cd00;\">\"course_points\"<\/span>]:\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'  &lt;wpt lat=\"{poi[\"y\"]}\" lon=\"{poi[\"x\"]}\"&gt;'<\/span>)\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'    &lt;name&gt;{quote_xml(poi[\"n\"])}&lt;\/name&gt;'<\/span>)\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'  &lt;\/wpt&gt;'<\/span>)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"  &lt;trk&gt;&lt;trkseg&gt;\"<\/span>)\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> pt <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> data[<span style=\"color: #00cd00;\">'track_points'<\/span>]:\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">'    &lt;trkpt lat=\"{pt[\"y\"]}\" lon=\"{pt[\"x\"]}\"&gt;&lt;ele&gt;{pt[\"e\"]}&lt;\/ele&gt;&lt;\/trkpt&gt;'<\/span>)\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"  &lt;\/trkseg&gt;&lt;\/trk&gt;\"<\/span>)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"&lt;\/gpx&gt;\"<\/span>)\n<\/pre>\n<\/div>\n\n<p>\nYou invoke it by downloading the route and feeding it into the script:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-sh\">curl -s https:\/\/ridewithgps.com\/routes\/54493422.json | .\/ridewithgps-to-gpx.py &gt; out.gpx\n<\/pre>\n<\/div>\n\n<p>\nNote that the route number 54493422 is in the url above.<\/p>\n"},{"title":"mrcal 2.5 released!","author":{"name":"Dima Kogan"},"link":{"@attributes":{"href":"http:\/\/notes.secretsauce.net\/notes\/2026\/01\/18_mrcal-25-released.html"}},"updated":"2026-01-18T16:00:00Z","published":"2026-01-18T16:00:00Z","id":"notes\/2026\/01\/18_mrcal-25-released.html","category":{"@attributes":{"scheme":"\/tags\/mrcal.html","term":"mrcal","label":"mrcal"}},"content":"<p>\n<a href=\"https:\/\/mrcal.secretsauce.net\/\">mrcal<\/a> 2.5 is out: <a href=\"https:\/\/mrcal.secretsauce.net\/news-2.5.html\">the release notes<\/a>. Once <i>again<\/i>, this is mostly a bug-fix\nrelease en route to the big new features coming in 3.0.\n<\/p>\n\n<p>\nOne cool thing is that these tools have now matured enough to no longer be\nconsidered experimental. They have been used with great success in <i>lots<\/i> of\ncontexts across many different projects and organizations. Some highlights:\n<\/p>\n\n<ul class=\"org-ul\">\n<li>I've calibrated extremely wide lenses\n<\/li>\n<li>and extremely narrow lenses\n<\/li>\n<li>and joint systems containing many different kinds of lenses\n<\/li>\n<li>with lots of cameras at the same time. The biggest single joint calibration\nI've done today had 10 cameras, but I'll almost certainly encounter bigger\nsystems in the future\n<\/li>\n<li>mrcal has been used to process both visible and thermal cameras\n<\/li>\n<li>The new triangulated-feature capability has been used in a\nstructure-from-motion context to compute the world geometry on-line.\n<\/li>\n<li>mrcal has been used with weird experimental setups employing custom\ncalibration objects and single-view solves\n<\/li>\n<li>mrcal has calibrated <a href=\"https:\/\/github.com\/dkogan\/camera-lidar-calibration\/\">joint camera-LIDAR systems<\/a>\n<\/li>\n<li>and <a href=\"https:\/\/www.github.com\/dkogan\/mrcal\/blob\/master\/analyses\/calibrate-camera-imu.py\">joint camera-IMU systems<\/a>\n<\/li>\n<li>Lots of students use mrcal as part of <a href=\"https:\/\/photonvision.org\/\">PhotonVision<\/a>, the toolkit used by teams\nin the <a href=\"https:\/\/www.firstinspires.org\/robotics\/frc\">FIRST Robotics Competition<\/a>\n<\/li>\n<\/ul>\n\n<p>\nSome of the above is new, and not yet fully polished and documented and tested,\nbut it works.\n<\/p>\n\n<p>\nIn mrcal 2.5, <i>most<\/i> of the implementation of some new big features is written\nand committed, but it's still incomplete. The new stuff is there, but is lightly\ntested and documented. This will be completed eventually in mrcal 3.0:\n<\/p>\n\n<ul class=\"org-ul\">\n<li><a href=\"https:\/\/mrcal.secretsauce.net\/uncertainty-cross-reprojection.html\">Cross-reprojection uncertainty<\/a>, to be able to perform full calibrations with a\nsplined model and <i>without<\/i> a chessboard. <a href=\"https:\/\/mrcal.secretsauce.net\/mrcal-show-projection-uncertainty.html\"><code>mrcal-show-projection-uncertainty\n  --method cross-reprojection-rrp-Jfp<\/code><\/a> is available today, and works in the\nusual moving-chessboard-stationary camera case. Fully boardless coming later.\n<\/li>\n<li>More general view of uncertainty and diffs. I want to support extrinsics-only\nand\/or intrinsics computations-only in lots of scenarios. Uncertainty in point\nsolves is already available in some conditions, for instance if the points are\nfixed. New <a href=\"https:\/\/mrcal.secretsauce.net\/mrcal-show-stereo-pair-diff.html\"><code>mrcal-show-stereo-pair-diff<\/code> tool<\/a> reports an extrinsics+intrinsics\ndiff between two calibrations of a stereo pair; experimental\n<a href=\"https:\/\/www.github.com\/dkogan\/mrcal\/blob\/master\/analyses\/extrinsics-stability.py\"><code>analyses\/extrinsics-stability.py<\/code><\/a> tool reports an extrinsics-only diff. These\nare in contrast to the intrinsics-only uncertainty and diffs in the existing\n<a href=\"https:\/\/mrcal.secretsauce.net\/mrcal-show-projection-diff.html\"><code>mrcal-show-projection-diff<\/code><\/a> and <a href=\"https:\/\/mrcal.secretsauce.net\/mrcal-show-projection-uncertainty.html\"><code>mrcal-show-projection-uncertainty<\/code><\/a> tools.\nSome documentation in the <a href=\"https:\/\/mrcal.secretsauce.net\/uncertainty.html\">uncertainty<\/a> and <a href=\"https:\/\/mrcal.secretsauce.net\/differencing.html\">differencing<\/a> pages.\n<\/li>\n<li>Implicit point solves, using the triangulation routines in the optimization\ncost function. Should produce much more efficient structure-from-motion\nsolves. This is all the \"triangulated-features\" stuff. The cost function is\nprimarily built around <code>_mrcal_triangulated_error()<\/code>. This is demoed in\n<a href=\"https:\/\/www.github.com\/dkogan\/mrcal\/blob\/master\/test\/test-sfm-triangulated-points.py\"><code>test\/test-sfm-triangulated-points.py<\/code><\/a>. And I've been using\n<code>_mrcal_triangulated_error()<\/code> in structure-from-motion implementations within\nother optimization routines.\n<\/li>\n<\/ul>\n\n<p>\nmrcal is quite good already, and will be even better in the future. Try it\ntoday!\n<\/p>\n"},{"title":"Meshroom packaged for Debian","author":{"name":"Dima Kogan"},"link":{"@attributes":{"href":"http:\/\/notes.secretsauce.net\/notes\/2026\/01\/08_meshroom-packaged-for-debian.html"}},"updated":"2026-01-08T15:34:00Z","published":"2026-01-08T15:34:00Z","id":"notes\/2026\/01\/08_meshroom-packaged-for-debian.html","category":[{"@attributes":{"scheme":"\/tags\/tools.html","term":"tools","label":"tools"}},{"@attributes":{"scheme":"\/tags\/debian.html","term":"debian","label":"debian"}}],"content":"<p>\nLike the title says, I just packaged <a href=\"https:\/\/alicevision.org\/#meshroom\">Meshroom<\/a> (and all the adjacent\ndependencies) for Debian! This is a fancy photogrammetry toolkit that uses\nmodern software development methods. \"Modern\" meaning that it has a multitude of\ndependencies that come from lots of disparate places, which make it impossible\nfor a mere mortal to build the thing. The Linux \"installer\" is 13GB and probably\nis some sort of container, or something.\n<\/p>\n\n<p>\nBut now, if you have a Debian\/sid box with the non-free repos enabled, you can\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-sh\">sudo apt install meshroom\n<\/pre>\n<\/div>\n\n<p>\nAnd then you can generate and 3D-print a life-size, geometrically-accurate\nstatue of your cat. The <code>colmap<\/code> package does a similar thing, and has been in\nDebian for a while. I <i>think<\/i> it can't do as many things, but it's good to have\nboth tools easily available.\n<\/p>\n\n<p>\nThese packages are all in contrib, because they depend on a number of non-free\nthings, most notably CUDA.\n<\/p>\n\n<p>\nThis is currently in Debian\/sid, but should be picked up by the downstream\ndistros as they're released. The next noteworthy one is Ubuntu 26.04. Testing\nand feedback welcome.\n<\/p>\n"},{"title":"Using libpython3 without linking it in; and old Python, g++ compatibility patches","author":{"name":"Dima Kogan"},"link":{"@attributes":{"href":"http:\/\/notes.secretsauce.net\/notes\/2026\/01\/01_using-libpython3-without-linking-it-in-and-old-python-g-compatibility-patches.html"}},"updated":"2026-01-01T21:52:00Z","published":"2026-01-01T21:52:00Z","id":"notes\/2026\/01\/01_using-libpython3-without-linking-it-in-and-old-python-g-compatibility-patches.html","content":"<p>\nI just released <a href=\"https:\/\/mrcal.secretsauce.net\">mrcal 2.5<\/a>; much more about that in a future post. Here, I'd like\nto talk about some implementation details.\n<\/p>\n\n<div id=\"outline-container-sec-1\" class=\"outline-2\">\n<h2 id=\"sec-1\">libpython3 and linking<\/h2>\n<div class=\"outline-text-2\" id=\"text-1\">\n<div class=\"alert alert-info\"><p class=\"alert-heading\">Follow-up patches<\/p>\n\n<p>\nThe technique described here ended up still incomplete! These extra patches were\nneeded:\n<\/p>\n\n<pre class=\"example\">\ncommit c2475520ff0e4905e5d4b2f251ccbd0d54b7e02c\nAuthor: Dima Kogan &lt;dima@secretsauce.net&gt;\nDate:   Mon Jan 19 12:44:15 2026 -0800\n\n    I weaken the python symbols using __asm__\n    \n    The previous scheme didn't work everywhere. Before this I had\n    \n      #include &lt;Python.h&gt;\n      extern __typeof__(Py_xxx) Py_xxx __attribute__((weak));\n      ...\n    \n    This worked for most functions and for all functions with gcc on Linux. With\n    clang or any compiler on macos this did not work for inline functions. For\n    instance, this test program would NOT get a weakened f2():\n    \n      int f(int);\n      int f2(int);\n      static inline int f3(int x)\n      {\n          return f2(x);\n      }\n      extern int f  (int) __attribute__((weak));\n      extern int f2 (int) __attribute__((weak));\n      int g(int x)\n      {\n          return f(x) + f3(x) + 5;\n      }\n    \n    I would need to extern int f2 (int) __attribute__((weak)); prior to #including\n    &lt;Python.h&gt;, but that wasn't possible because I didn't have the type of f2 then.\n    The solution is to use __asm__ to tell the linker directly about the weak\n    symbol. This does not need the prototype, and could be done before #include\n    &lt;Python.h&gt;. This works. The only other wrinkle is a different attribute name on\n    Linux and macos, and this is good now.\n\ncommit 1f8673c0374bcd0588bba17ce8f2bbc55db040cf\nAuthor: Dima Kogan &lt;dima@secretsauce.net&gt;\nDate:   Thu Jan 8 11:29:00 2026 -0800\n\n    python-cameramodel-converter uses the limited python api if possible part 2\n    \n    Needed more logic to look at the python version before making a decision about\n    which API to pull in\n\ncommit 9ae47f4feeec02df87a2364ba5bede4802259cd0\nAuthor: Dima Kogan &lt;dima@secretsauce.net&gt;\nDate:   Thu Jan 8 10:58:16 2026 -0800\n\n    python-cameramodel-converter uses the limited python api if possible\n    \n    I'm not linking against libpython, so I'm not tying myself to any particular\n    ABI version of libpython. To make this work regardless, I use the \"limited\"\n    API:\n    \n      https:\/\/docs.python.org\/3\/c-api\/stable.html#stable\n    \n    I'm calling PyUnicode_AsUTF8AndSize(), which entered the limited API in 3.10. I\n    still allow this to build in older Python, but I don't explicitly declare the\n    limited api. In that case, it is possible that using the binary libmrcal.so\n    built with &lt;3.10 with a later Python might cause issues\n\ncommit 592303d6d90712ddbde4aa7a152871d5b602cfdb\nAuthor: Dima Kogan &lt;dima@secretsauce.net&gt;\nDate:   Thu Jan 8 10:50:44 2026 -0800\n\n    python-cameramodel-converter only functions that are in the limited api &gt;= 3.10\n<\/pre>\n\n\n<\/div>\n\n\n<p>\nmrcal is a C library and a Python library. Much of mrcal itself interfaces the C\nand Python libraries. And it is common for external libraries to want to pass\nPython <code>mrcal.cameramodel<\/code> objects to <i>their<\/i> C code. The obvious way to do this\nis in <a href=\"https:\/\/docs.python.org\/3\/c-api\/arg.html#other-objects\">a <i>converter<\/i> function<\/a> in an <code>O&amp;<\/code> argument to\n<a href=\"https:\/\/docs.python.org\/3\/c-api\/arg.html#c.PyArg_ParseTupleAndKeywords\"><code>PyArg_ParseTupleAndKeywords()<\/code><\/a>. I <a href=\"https:\/\/github.com\/dkogan\/mrcal\/blob\/master\/python-cameramodel-converter.h\">wrote this <code>mrcal_cameramodel_converter()<\/code>\nfunction<\/a>, which opened a whole can of worms when thinking about the compiling\nand linking and distribution of this thing.\n<\/p>\n\n<p>\n<code>mrcal_cameramodel_converter()<\/code> is meant to be called by code that implements\nPython-wrapping of C code. This function will be called by the\n<a href=\"https:\/\/docs.python.org\/3\/c-api\/arg.html#c.PyArg_ParseTupleAndKeywords\"><code>PyArg_ParseTupleAndKeywords()<\/code><\/a> Python library function, and it uses the Python\nC API itself. Since it uses the Python C API, it would normally link against\n<code>libpython<\/code>. However:\n<\/p>\n\n<ul class=\"org-ul\">\n<li>The natural place to distribute this is in <code>libmrcal.so<\/code>, but <i>this<\/i> library\ndoesn't touch Python, and I'd rather not pull in all of <code>libpython<\/code> for this\nutility function, even in the 99% case when that function won't even be called\n<\/li>\n<li>In <a href=\"https:\/\/github.com\/dkogan\/mrcal\/issues\/22\">some cases<\/a> linking to <code>libpython<\/code> actually breaks things, so I never do\nthat anymore anyway. This is fine: since this code will only ever be called by\n<code>libpython<\/code> itself, we're guaranteed that <code>libpython<\/code> will already be loaded,\nand we don't need to ask for it.\n<\/li>\n<\/ul>\n\n<p>\nOK, let's not link to <code>libpython<\/code> then. But if we do that, we're going to have\nunresolved references to our <code>libpython<\/code> calls, and the loader will complain\nwhen loading <code>libmrcal.so<\/code>, even if we're not actually calling those functions.\nThis has an obvious solution: the references to the <code>libpython<\/code> calls should be\nmarked weak. That won't generate unresolved-reference errors, and everything\nwill be great.\n<\/p>\n\n<p>\nOK, how do we mark things weak? There're two usual methods:\n<\/p>\n\n<ol class=\"org-ol\">\n<li>We mark the declaration (or definition?) or the relevant functions with\n<code>__attribute__((weak))<\/code>\n<\/li>\n\n<li>We weaken the symbols after the compile with <code>objcopy --weaken<\/code>.\n<\/li>\n<\/ol>\n\n<p>\nMethod 1 is more work: I don't want to keep track of what Python API calls I'm\nactually making. This is non-trivial, because some of the <code>Py_...()<\/code> invocations\nin my code are actually macros that call functions internally that I must\nweaken. Furthermore, all the functions are declared in <code>Python.h<\/code> that I don't\ncontrol. I can re-declare stuff with <code>__attribute__((weak))<\/code>, but then I have to\nmatch the prototypes. And I have to hope that re-declaring these will make\n<code>__attribute__((weak))<\/code> actually work.\n<\/p>\n\n<p>\nSo clearly I want method 2. I implemented it:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-makefile\"><span style=\"color: #0000ee; font-weight: bold;\">python-cameramodel-converter.o<\/span>: %.o:%.c\n        $(<span style=\"color: #cdcd00;\">c_build_rule<\/span>); mv <span style=\"color: #0000ee; font-weight: bold;\">$<\/span><span style=\"color: #cd00cd; font-weight: bold;\">@<\/span> _<span style=\"color: #0000ee; font-weight: bold;\">$<\/span><span style=\"color: #cd00cd; font-weight: bold;\">@<\/span>\n        $(<span style=\"color: #cdcd00;\">OBJCOPY<\/span>) --wildcard --weaken-symbol=<span style=\"color: #00cd00;\">'Py*'<\/span> --weaken-symbol=<span style=\"color: #00cd00;\">'_Py*'<\/span> _<span style=\"color: #0000ee; font-weight: bold;\">$<\/span><span style=\"color: #cd00cd; font-weight: bold;\">@<\/span> <span style=\"color: #0000ee; font-weight: bold;\">$<\/span><span style=\"color: #cd00cd; font-weight: bold;\">@<\/span>\n<\/pre>\n<\/div>\n\n<p>\nWorks great on my machine! But doesn't work on other people's machines. Because\nonly the most recent <code>objcopy<\/code> tool actually works to weaken <i>references<\/i>.\nApparently the older tools only weaken definitions, which isn't useful to me,\nand the tool only started handling references very recently.\n<\/p>\n\n<p>\nWell that sucks. I guess I will need to mark the symbols with\n<code>__attribute__((weak))<\/code> after all. I use the <code>nm<\/code> tool to find the symbols that\nshould be weakened, and I apply the attribute with this macro:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-c\"><span style=\"color: #0000ee; font-weight: bold;\">#define<\/span> <span style=\"color: #0000ee; font-weight: bold;\">WEAKEN<\/span>(<span style=\"color: #cdcd00;\">f<\/span>) <span style=\"color: #00cdcd; font-weight: bold;\">extern<\/span> <span style=\"color: #0000ee; font-weight: bold;\">__typeof__<\/span>(f) <span style=\"color: #00cd00;\">f<\/span> <span style=\"color: #00cdcd; font-weight: bold;\">__attribute__<\/span>((weak));\n<\/pre>\n<\/div>\n\n<p>\nThe prototypes are handled by <code>__typeof__<\/code>. So are we done? With gcc, we are\ndone. With clang we are not done. Apparently this macro does not weaken symbols\ngenerated by inline function calls if using clang I have no idea if this is a\nbug. The Python internal machinery has some of these, so this doesn't weaken\n<i>all<\/i> the symbols. I give up on the people that both have a too-old objcopy\n<i>and<\/i> are using clang, and declare victory. So the logic ends up being:\n<\/p>\n\n<ol class=\"org-ol\">\n<li>Compile\n<\/li>\n<li><code>objcopy --weaken<\/code>\n<\/li>\n<li><code>nm<\/code> to find the non-weak Python references\n<\/li>\n<li>If there aren't any, our <code>objcopy<\/code> call worked and we're done!\n<\/li>\n<li>Otherwise, compile again, but explicitly asking to weaken <i>those<\/i> symbols\n<\/li>\n<li><code>nm<\/code> again to see if the compiler didn't do it\n<\/li>\n<li>If any non-weak references still remain, complain and give up.\n<\/li>\n<\/ol>\n\n<p>\nWhew. This logic appears <a href=\"https:\/\/github.com\/dkogan\/mrcal\/blob\/98a0ae4bce9eede6ab86b98975c322cc475b60fc\/Makefile#L61\">here<\/a> and <a href=\"https:\/\/github.com\/dkogan\/mrcal\/blob\/98a0ae4bce9eede6ab86b98975c322cc475b60fc\/python-cameramodel-converter.c#L19\">here<\/a>. There were <i>even more<\/i> things to deal\nwith here: calling <code>nm<\/code> and <code>objcopy<\/code> needed special attention and build-system\nsupport in case we were cross-building. I <a href=\"https:\/\/github.com\/dkogan\/mrbuild\/commit\/8ef5d4de06684535c9b3c09612492963af25ea45\">took care of it in <code>mrbuild<\/code><\/a>.\n<\/p>\n\n<p>\nThis worked for a while. Until the converter code started to fail. Because &#x2026;.\n<\/p>\n<\/div>\n<\/div>\n\n<div id=\"outline-container-sec-2\" class=\"outline-2\">\n<h2 id=\"sec-2\">Supporting old Python<\/h2>\n<div class=\"outline-text-2\" id=\"text-2\">\n<p>\n&#x2026;. I was using <code>PyTuple_GET_ITEM()<\/code>. This is a macro to access <code>PyTupleObject<\/code>\ndata. So the layout of <code>PyTupleObject<\/code> ended up encoded in <code>libmrcal.so<\/code>. But\napparently this wasn't stable, and changed between Python3.13 and Python3.14. As\ndescribed above, I'm not linking to <code>libpython<\/code>, so there's no <code>NEEDED<\/code> tag to\nmake sure we pull in the right version. The solution was to <a href=\"https:\/\/github.com\/dkogan\/mrcal\/commit\/a922e354ec4521580ed73812ed61b04312c29411\">call the\n<code>PyTuple_GetItem()<\/code> function instead<\/a>. This is unsatisfying, and means that in\ntheory other stuff here might stop working in some Python 3.future, but I'm\nready to move on for now.\n<\/p>\n\n<p>\nThere were other annoying gymnastics that had to be performed to make this work\nwith old-but-not-super old tooling.\n<\/p>\n\n<p>\nThe Python people deprecated <a href=\"https:\/\/docs.python.org\/3\/c-api\/module.html#c.PyModule_AddObject\"><code>PyModule_AddObject()<\/code><\/a>, and added <a href=\"https:\/\/docs.python.org\/3\/c-api\/module.html#c.PyModule_Add\"><code>PyModule_Add()<\/code><\/a>\nas a replacement. I want to support Pythons before and after this happened, so I\nneeded some <a href=\"https:\/\/github.com\/dkogan\/mrcal\/blob\/98a0ae4bce9eede6ab86b98975c322cc475b60fc\/mrcal-pywrap.c#L4513\">if statements<\/a>. Today the old function still works, but eventually it\nwill stop, and I will have needed to do this typing sooner or later.\n<\/p>\n<\/div>\n<\/div>\n\n<div id=\"outline-container-sec-3\" class=\"outline-2\">\n<h2 id=\"sec-3\">Supporting old C++ compilers<\/h2>\n<div class=\"outline-text-2\" id=\"text-3\">\n<p>\nmrcal is a C project, but it is common for people to want to <code>#include<\/code> the\nheaders from C++. I widely use C99 designated initializers (27-years old in C!),\nwhich causes issues with not-very-old C++ compilers. I worked around this\ninitialization <a href=\"https:\/\/github.com\/dkogan\/mrcal\/commit\/e949893348dbe915e66b817fb821b8b9b818fa89\">in one spot<\/a>, and disabled it a feature for a too-old compiler <a href=\"https:\/\/github.com\/dkogan\/mrcal\/commit\/88d81d6dd7f259c5266002e6c6ce9d2d1616575e\">in\nanother spot<\/a>. Fortunately, semi-recent tooling supports my usages, so this is\nbecoming a non-issue as time goes on.\n<\/p>\n<\/div>\n<\/div>\n"},{"title":"Eigen macro specializations crashes","author":{"name":"Dima Kogan"},"link":{"@attributes":{"href":"http:\/\/notes.secretsauce.net\/notes\/2025\/03\/17_eigen-macro-specializations-crashes.html"}},"updated":"2025-03-17T20:52:00Z","published":"2025-03-17T20:52:00Z","id":"notes\/2025\/03\/17_eigen-macro-specializations-crashes.html","category":{"@attributes":{"scheme":"\/tags\/tools.html","term":"tools","label":"tools"}},"content":"<p>\nThere's an issue in the <a href=\"https:\/\/eigen.tuxfamily.org\/index.php?title=Main_Page\">Eigen linear algebra library<\/a> where linking together\nobjects compiled with different flags causes the resulting binary to crash. Some\ndetails are written-up in <a href=\"https:\/\/www.mail-archive.com\/debian-science@lists.debian.org\/msg13672.html\">this mailing list thread<\/a>.\n<\/p>\n\n<p>\nI just encountered a situation where a large application sometimes crashes for\nunknown reasons, and needed a method to determine whether this Eigen issue could\nbe the cause. I ended up doing this by using the DWARF data to see if the linked\nbinary contains the different incompatible flavors of <code>malloc<\/code> \/ <code>free<\/code> or not.\n<\/p>\n\n<p>\nI downloaded the <a href=\"https:\/\/www.mail-archive.com\/debian-science@lists.debian.org\/msg13710.html\">small demo program showing the problem<\/a>. I built it:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-sh\"><span style=\"color: #cdcd00;\">CCXXXFLAGS<\/span>=-g make\n<\/pre>\n<\/div>\n\n<p>\nHere if you run <code>.\/main<\/code>, the bug is triggered, and a crash occurs. I looked at\nthe debug info for the code in question:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-sh\"><span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> o (main lib.so) {\n  <span style=\"color: #0000ee; font-weight: bold;\">echo<\/span> <span style=\"color: #00cd00;\">\"======== $o\"<\/span>;\n  readelf --debug-dump=decodedline $<span style=\"color: #cdcd00;\">o<\/span> <span style=\"color: #00cd00;\">\\<\/span>\n  | awk <span style=\"color: #00cd00;\">\\<\/span>\n    <span style=\"color: #00cd00;\">'$1 ~ \/^Memory.h\/<\/span>\n<span style=\"color: #00cd00;\">     {<\/span>\n<span style=\"color: #00cd00;\">       if(180 &lt;= $2 &amp;&amp; $2 &lt;= 186) {<\/span>\n<span style=\"color: #00cd00;\">         have[\"malloc_glibc\"]=1<\/span>\n<span style=\"color: #00cd00;\">       }<\/span>\n<span style=\"color: #00cd00;\">       if(188 == $2) {<\/span>\n<span style=\"color: #00cd00;\">         have[\"malloc_handmade\"]=1<\/span>\n<span style=\"color: #00cd00;\">       }<\/span>\n<span style=\"color: #00cd00;\">       if(201 &lt;= $2 &amp;&amp; $2 &lt;= 204) {<\/span>\n<span style=\"color: #00cd00;\">         have[\"free_glibc\"]=1<\/span>\n<span style=\"color: #00cd00;\">       }<\/span>\n<span style=\"color: #00cd00;\">       if(206 == $2) {<\/span>\n<span style=\"color: #00cd00;\">         have[\"free_handmade\"]=1<\/span>\n<span style=\"color: #00cd00;\">       }<\/span>\n<span style=\"color: #00cd00;\">     }<\/span>\n<span style=\"color: #00cd00;\">     END<\/span>\n<span style=\"color: #00cd00;\">     {<\/span>\n<span style=\"color: #00cd00;\">       for (var in have) {<\/span>\n<span style=\"color: #00cd00;\">         print(var);<\/span>\n<span style=\"color: #00cd00;\">       }<\/span>\n<span style=\"color: #00cd00;\">     }'<\/span>\n}\n<\/pre>\n<\/div>\n\n<p>\nIt says:\n<\/p>\n\n<pre class=\"example\">\n======== main\nfree_handmade\n======== lib.so\nmalloc_glibc\nfree_glibc\n<\/pre>\n\n<p>\nHere I looked at <code>main<\/code> and <code>lib.so<\/code> (the build products from this little demo).\nIn a real case you'd look at every shared library linked into the binary and the\nbinary itself. On my machine <code>\/usr\/include\/eigen3\/Eigen\/src\/Core\/util\/Memory.h<\/code>\nlooks like this, starting on line 174:\n<\/p>\n\n<pre class=\"example\">\n174 EIGEN_DEVICE_FUNC inline void* aligned_malloc(std::size_t size)\n175 {\n176   check_that_malloc_is_allowed();\n177 \n178   void *result;\n179   #if (EIGEN_DEFAULT_ALIGN_BYTES==0) || EIGEN_MALLOC_ALREADY_ALIGNED\n180 \n181     EIGEN_USING_STD(malloc)\n182     result = malloc(size);\n183 \n184     #if EIGEN_DEFAULT_ALIGN_BYTES==16\n185     eigen_assert((size&lt;16 || (std::size_t(result)%16)==0) &amp;&amp; \"System's malloc returned an unaligned pointer. Compile with EIGEN_MALLOC_ALREADY_ALIGNED=0 to fallback to handmade aligned memory allocator.\");\n186     #endif\n187   #else\n188     result = handmade_aligned_malloc(size);\n189   #endif\n190 \n191   if(!result &amp;&amp; size)\n192     throw_std_bad_alloc();\n193 \n194   return result;\n195 }\n196 \n197 \/** \\internal Frees memory allocated with aligned_malloc. *\/\n198 EIGEN_DEVICE_FUNC inline void aligned_free(void *ptr)\n199 {\n200   #if (EIGEN_DEFAULT_ALIGN_BYTES==0) || EIGEN_MALLOC_ALREADY_ALIGNED\n201 \n202     EIGEN_USING_STD(free)\n203     free(ptr);\n204 \n205   #else\n206     handmade_aligned_free(ptr);\n207   #endif\n208 }\n<\/pre>\n\n<p>\nThe above <code>awk<\/code> script looks at the two malloc paths and the two free paths, and\nwe can clearly see that it only ever calls <code>malloc_glibc()<\/code>, but has both\nflavors of <code>free()<\/code>. So this can crash. We want to see that the whole executable\n(shared libraries and all) should only have one type of <code>malloc()<\/code> and <code>free()<\/code>,\nand that would guarantee no crashing.\n<\/p>\n\n<p>\nThere are a more functions in that header that should be instrumented\n(<code>realloc()<\/code> for instance) and the different alignment paths should be\ninstrumented similarly (as described in the mailing list thread above), but here\nwe see that this technique works.\n<\/p>\n"},{"title":"Getting precise timings out of RS-232 output","author":{"name":"Dima Kogan"},"link":{"@attributes":{"href":"http:\/\/notes.secretsauce.net\/notes\/2025\/03\/14_getting-precise-timings-out-of-rs-232-output.html"}},"updated":"2025-03-14T12:47:00Z","published":"2025-03-14T12:47:00Z","id":"notes\/2025\/03\/14_getting-precise-timings-out-of-rs-232-output.html","category":{"@attributes":{"scheme":"\/tags\/tools.html","term":"tools","label":"tools"}},"content":"<p>\nFor uninteresting reasons I need very regular 58Hz pulses coming out of an\nRS-232 Tx line: the time between each pulse should be as close to 1\/58s as\npossible. I produce each pulse by writing an <code>\\xFF<\/code> byte to the device. The\nstart bit is the only active-voltage bit being sent, and that produces my pulse.\nI wrote this obvious C program:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-c\"><span style=\"color: #0000ee; font-weight: bold;\">#include<\/span> <span style=\"color: #00cd00;\">&lt;stdio.h&gt;<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">#include<\/span> <span style=\"color: #00cd00;\">&lt;stdlib.h&gt;<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">#include<\/span> <span style=\"color: #00cd00;\">&lt;stdbool.h&gt;<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">#include<\/span> <span style=\"color: #00cd00;\">&lt;sys\/ioctl.h&gt;<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">#include<\/span> <span style=\"color: #00cd00;\">&lt;unistd.h&gt;<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">#include<\/span> <span style=\"color: #00cd00;\">&lt;fcntl.h&gt;<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">#include<\/span> <span style=\"color: #00cd00;\">&lt;termios.h&gt;<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">#include<\/span> <span style=\"color: #00cd00;\">&lt;stdint.h&gt;<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">#include<\/span> <span style=\"color: #00cd00;\">&lt;sys\/time.h&gt;<\/span>\n\n<span style=\"color: #00cdcd; font-weight: bold;\">static<\/span> <span style=\"color: #00cd00;\">uint64_t<\/span> <span style=\"color: #0000ee; font-weight: bold;\">gettimeofday_uint64<\/span>()\n{\n    <span style=\"color: #00cdcd; font-weight: bold;\">struct<\/span> <span style=\"color: #00cd00;\">timeval<\/span> <span style=\"color: #cdcd00;\">tv<\/span>;\n    gettimeofday(&amp;tv, <span style=\"color: #cd00cd;\">NULL<\/span>);\n    <span style=\"color: #00cdcd; font-weight: bold;\">return<\/span> (<span style=\"color: #00cd00;\">uint64_t<\/span>) tv.tv_sec * 1000000ULL + (<span style=\"color: #00cd00;\">uint64_t<\/span>) tv.tv_usec;\n}\n\n<span style=\"color: #00cd00;\">int<\/span> <span style=\"color: #0000ee; font-weight: bold;\">main<\/span>(<span style=\"color: #00cd00;\">int<\/span> <span style=\"color: #cdcd00;\">argc<\/span>, <span style=\"color: #00cd00;\">char<\/span>* <span style=\"color: #cdcd00;\">argv<\/span>[])\n{\n    <span style=\"color: #cdcd00;\">\/\/ <\/span><span style=\"color: #cdcd00;\">open the serial device, and make it as raw as possible<\/span>\n    <span style=\"color: #00cdcd; font-weight: bold;\">const<\/span> <span style=\"color: #00cd00;\">char<\/span>* <span style=\"color: #cdcd00;\">device<\/span> = <span style=\"color: #00cd00;\">\"\/dev\/ttyS0\"<\/span>;\n    <span style=\"color: #00cdcd; font-weight: bold;\">const<\/span> <span style=\"color: #00cd00;\">speed_t<\/span> <span style=\"color: #cdcd00;\">baud<\/span> = B9600;\n\n    <span style=\"color: #00cd00;\">int<\/span> <span style=\"color: #cdcd00;\">fd<\/span> = open(device, O_WRONLY|O_NOCTTY);\n    tcflush(fd, TCIOFLUSH);\n\n    <span style=\"color: #00cdcd; font-weight: bold;\">struct<\/span> <span style=\"color: #00cd00;\">termios<\/span> <span style=\"color: #cdcd00;\">options<\/span> = {.c_iflag = IGNBRK,\n                              .c_cflag = CS8 | CREAD | CLOCAL};\n    cfsetspeed(&amp;options, baud);\n    tcsetattr(fd, TCSANOW, &amp;options);\n\n    <span style=\"color: #00cdcd; font-weight: bold;\">const<\/span> <span style=\"color: #00cd00;\">uint64_t<\/span> <span style=\"color: #cdcd00;\">T_us<\/span> = (<span style=\"color: #00cd00;\">uint64_t<\/span>)(1e6 \/ 58.);\n\n    <span style=\"color: #00cdcd; font-weight: bold;\">const<\/span> <span style=\"color: #00cd00;\">uint64_t<\/span> <span style=\"color: #cdcd00;\">t0<\/span> = gettimeofday_uint64();\n    <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span>(<span style=\"color: #00cd00;\">int<\/span> <span style=\"color: #cdcd00;\">i<\/span>=0; ; i++)\n    {\n        <span style=\"color: #00cdcd; font-weight: bold;\">const<\/span> <span style=\"color: #00cd00;\">uint64_t<\/span> <span style=\"color: #cdcd00;\">t_target<\/span> = t0 + T_us*i;\n        <span style=\"color: #00cdcd; font-weight: bold;\">const<\/span> <span style=\"color: #00cd00;\">uint64_t<\/span> <span style=\"color: #cdcd00;\">t1<\/span>       = gettimeofday_uint64();\n\n        <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span>(t_target &gt; t1)\n            usleep(t_target - t1);\n\n        write(fd, &amp;((<span style=\"color: #00cd00;\">char<\/span>){<span style=\"color: #00cd00;\">'\\xff'<\/span>}), 1);\n    }\n    <span style=\"color: #00cdcd; font-weight: bold;\">return<\/span> 0;\n}\n<\/pre>\n<\/div>\n\n<p>\nThis tries to make sure that each <code>write()<\/code> call happens at 58Hz. I need these\npulses to be regular, so I need to also make sure that the time between each\nuserspace <code>write()<\/code> and when the edge actually hits the line is as short as\npossible or, at least, stable.\n<\/p>\n\n<p>\nPotential reasons for timing errors:\n<\/p>\n\n<ol class=\"org-ol\">\n<li>The <code>usleep()<\/code> doesn't wake up exactly when it should. This is subject to the\nLinux scheduler waking up the <code>trigger<\/code> process\n<\/li>\n<li>The <code>write()<\/code> almost certainly ends up scheduling a helper task to actually\nwrite the <code>\\xFF<\/code> to the hardware. This helper task is also subject to the\nLinux scheduler waking it up.\n<\/li>\n<li>Whatever the hardware does. RS-232 doesn't give you any guarantees about\nbyte-byte timings, so this could be an unfixable source of errors\n<\/li>\n<\/ol>\n\n<p>\nThe scheduler-related questions are observable without any extra hardware, so\nlet's do that first.\n<\/p>\n\n<p>\nI run the <code>.\/trigger<\/code> program, and look at diagnostics while that's running.\n<\/p>\n\n<p>\nI look at some device details:\n<\/p>\n\n<pre class=\"example\">\n# ls -lh \/dev\/ttyS0\ncrw-rw---- 1 root dialout 4, 64 Mar  6 18:11 \/dev\/ttyS0\n\n# ls -lh \/sys\/dev\/char\/4:64\/\ntotal 0\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 close_delay\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 closing_wait\n-rw-r--r-- 1 root root 4.0K Mar  6 16:51 console\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 custom_divisor\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 dev\nlrwxrwxrwx 1 root root    0 Mar  6 16:51 device -&gt; ..\/..\/..\/0000:00:16.3:0.0\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 flags\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 iomem_base\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 iomem_reg_shift\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 io_type\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 irq\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 line\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 port\ndrwxr-xr-x 2 root root    0 Mar  6 16:51 power\n-rw-r--r-- 1 root root 4.0K Mar  6 16:51 rx_trig_bytes\nlrwxrwxrwx 1 root root    0 Mar  6 16:51 subsystem -&gt; ..\/..\/..\/..\/..\/..\/..\/class\/tty\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 type\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 uartclk\n-rw-r--r-- 1 root root 4.0K Mar  6 16:51 uevent\n-r--r--r-- 1 root root 4.0K Mar  6 16:51 xmit_fifo_size\n<\/pre>\n\n<p>\nUnsurprisingly, this is a part of the <code>tty<\/code> subsystem. I don't want to spend the\ntime to really figure out how this works, so let me look at <i>all<\/i> the <code>tty<\/code>\nkernel calls and also at all the kernel tasks scheduled by the <code>trigger<\/code>\nprocess, since I suspect that the actual hardware poke is happening in a helper\ntask. I see this:\n<\/p>\n\n<pre class=\"example\">\n# bpftrace -e 'k:*tty* \/comm==\"trigger\"\/\n               { printf(\"%d %d %s\\n\",pid,tid,probe); }\n               t:sched:sched_wakeup \/comm==\"trigger\"\/\n               { printf(\"switching to %s(%d); current backtrace:\", args.comm, args.pid); print(kstack());  }'\n\n...\n\n3397345 3397345 kprobe:tty_ioctl\n3397345 3397345 kprobe:tty_check_change\n3397345 3397345 kprobe:__tty_check_change\n3397345 3397345 kprobe:tty_wait_until_sent\n3397345 3397345 kprobe:tty_write\n3397345 3397345 kprobe:file_tty_write.isra.0\n3397345 3397345 kprobe:tty_ldisc_ref_wait\n3397345 3397345 kprobe:n_tty_write\n3397345 3397345 kprobe:tty_hung_up_p\nswitching to kworker\/0:1(3400169); current backtrace:\n        ttwu_do_activate+268\n        ttwu_do_activate+268\n        try_to_wake_up+605\n        kick_pool+92\n        __queue_work.part.0+582\n        queue_work_on+101\n        rpm_resume+1398\n        __pm_runtime_resume+75\n        __uart_start+85\n        uart_write+150\n        n_tty_write+1012\n        file_tty_write.isra.0+373\n        vfs_write+656\n        ksys_write+109\n        do_syscall_64+130\n        entry_SYSCALL_64_after_hwframe+118\n\n3397345 3397345 kprobe:tty_update_time\n3397345 3397345 kprobe:tty_ldisc_deref\n\n... repeated with each pulse ...\n<\/pre>\n\n<p>\nLooking at the sources I see that <code>uart_write()<\/code> calls <code>__uart_start()<\/code>, which\nschedules a task to call <code>serial_port_runtime_resume()<\/code> which eventually calls\n<code>serial8250_tx_chars()<\/code>, which calls some low-level functions to actually send\nthe bits.\n<\/p>\n\n<p>\nI look at the time between two of those calls to quantify the scheduler latency:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-sh\"><span style=\"color: #cdcd00;\">pulserate<\/span>=58\n\nsudo zsh -c <span style=\"color: #00cd00;\">\\<\/span>\n  <span style=\"color: #00cd00;\">'( echo \"# dt_write_ns dt_task_latency_ns\";<\/span>\n<span style=\"color: #00cd00;\">     bpftrace -q -e \"k:vfs_write \/comm==\\\"trigger\\\" &amp;&amp; arg2==1\/<\/span>\n<span style=\"color: #00cd00;\">                     {\\$t=nsecs(); if(@t0) { @dt_write = \\$t-@t0; } @t0=\\$t;}<\/span>\n<span style=\"color: #00cd00;\">                     k:serial8250_tx_chars \/@dt_write\/<\/span>\n<span style=\"color: #00cd00;\">                     {\\$t=nsecs(); printf(\\\"%d %d\\\\n\\\", @dt_write, \\$t-@t0);}\"<\/span>\n<span style=\"color: #00cd00;\">   )'<\/span> <span style=\"color: #00cd00;\">\\<\/span>\n| vnl-filter                  <span style=\"color: #00cd00;\">\\<\/span>\n    --stream -p <span style=\"color: #cdcd00;\">dt_write_ms<\/span>=<span style=\"color: #00cd00;\">\"dt_write_ns\/1e6 - 1e3\/$pulserate\"<\/span>,<span style=\"color: #cdcd00;\">dt_task_latency_ms<\/span>=dt_task_latency_ns\/1e6 <span style=\"color: #00cd00;\">\\<\/span>\n| feedgnuplot  <span style=\"color: #00cd00;\">\\<\/span>\n    --stream   <span style=\"color: #00cd00;\">\\<\/span>\n    --lines    <span style=\"color: #00cd00;\">\\<\/span>\n    --points   <span style=\"color: #00cd00;\">\\<\/span>\n    --xlen 200 <span style=\"color: #00cd00;\">\\<\/span>\n    --vnl      <span style=\"color: #00cd00;\">\\<\/span>\n    --autolegend <span style=\"color: #00cd00;\">\\<\/span>\n    --xlabel <span style=\"color: #00cd00;\">'Pulse index'<\/span> <span style=\"color: #00cd00;\">\\<\/span>\n    --ylabel <span style=\"color: #00cd00;\">'Latency (ms)'<\/span>\n<\/pre>\n<\/div>\n\n<p>\nHere I'm making a realtime plot showing\n<\/p>\n\n<ul class=\"org-ul\">\n<li>The offset from 58Hz of when each <code>write()<\/code> call happens. This shows effect #1\nfrom above: how promptly the <code>trigger<\/code> process wakes up\n<\/li>\n<li>The latency of the helper task. This shows effect #2 above.\n<\/li>\n<\/ul>\n\n<p>\nThe raw data as I tweak things lives <a href=\"..\/..\/..\/notes\/2025\/03\/14_getting-precise-timings-out-of-rs-232-output\/timings.scheduler.vnl\">here<\/a>. Initially I see big latency spikes:\n<\/p>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2025\/03\/14_getting-precise-timings-out-of-rs-232-output\/timings.scheduler.1.noise.svg\" alt=\"timings.scheduler.1.noise.svg\" width=\"80%\" \/>\n<\/p>\n<\/div>\n\n<p>\nThese can be fixed by adjusting the priority of the <code>trigger<\/code> task. This tells\nthe scheduler to wake that task up <i>first<\/i>, even if something else is currently\nusing the CPU. I do this:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-sh\">sudo chrt -p 90 <span style=\"color: #cdcd00;\">`pidof trigger`<\/span>\n<\/pre>\n<\/div>\n\n<p>\nAnd I get better-looking latencies:\n<\/p>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2025\/03\/14_getting-precise-timings-out-of-rs-232-output\/timings.scheduler.2.clean.svg\" alt=\"timings.scheduler.2.clean.svg\" width=\"80%\" \/>\n<\/p>\n<\/div>\n\n<p>\nDuring some experiments (not in this dataset) I would see high helper-task\ntiming instabilities as well. These could be fixed by prioritizing the helper\ntask. In this kernel (<code>6.12<\/code>) the helper task is called <code>kworker\/N<\/code> where <code>N<\/code> is\nthe CPU index. I tie the <code>trigger<\/code> process to cpu 0, and priorities all the\nrelevant helpers:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-sh\">taskset -c 0 .\/trigger 58\n\npgrep -f kworker\/0 | <span style=\"color: #00cdcd; font-weight: bold;\">while<\/span> { <span style=\"color: #0000ee; font-weight: bold;\">read<\/span> pid } { sudo chrt -p 90 $<span style=\"color: #cdcd00;\">pid<\/span> }\n<\/pre>\n<\/div>\n\n<p>\nThis fixes the helper-task latency spikes.\n<\/p>\n\n<p>\nOK, so it looks like on the software side we're good to within 0.1ms of the true\nperiod. This is in the ballpark of the precision I need; even this might be too\nhigh. It's possible to try to push the software to do better: one could look at\nthe kernel sources a bit more, to do smarter things with priorities or to try an\n<code>-rt<\/code> kernel. But all this doesn't matter if the serial hardware adds\nunacceptable delays. Let's look.\n<\/p>\n\n<p>\nLet's look at it with a logic analyzer. I use a saleae logic analyzer with\n<a href=\"https:\/\/sigrok.org\/\">sigrok<\/a>. The tool spits out the samples as it gets them, and an <code>awk<\/code> script\nfinds the edges and reports the timings to give me a realtime plot.\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-sh\"><span style=\"color: #cdcd00;\">samplerate<\/span>=500000;\n<span style=\"color: #cdcd00;\">pulserate<\/span>=58.;\nsigrok-cli -c <span style=\"color: #cdcd00;\">samplerate<\/span>=$<span style=\"color: #cdcd00;\">samplerate<\/span> -O csv --continuous -C D1 <span style=\"color: #00cd00;\">\\<\/span>\n| mawk -Winteractive  <span style=\"color: #00cd00;\">\\<\/span>\n    <span style=\"color: #00cd00;\">\"prev_logic==0 &amp;&amp; \\$0==1 \\<\/span>\n<span style=\"color: #00cd00;\">     { <\/span>\n<span style=\"color: #00cd00;\">       iedge = NR;<\/span>\n<span style=\"color: #00cd00;\">       if(prev_iedge)<\/span>\n<span style=\"color: #00cd00;\">       {<\/span>\n<span style=\"color: #00cd00;\">         di = iedge -prev_iedge;<\/span>\n<span style=\"color: #00cd00;\">         dt = di\/$samplerate;<\/span>\n<span style=\"color: #00cd00;\">         print(dt*1000);<\/span>\n<span style=\"color: #00cd00;\">       }<\/span>\n<span style=\"color: #00cd00;\">       prev_iedge = iedge;<\/span>\n<span style=\"color: #00cd00;\">     }<\/span>\n<span style=\"color: #00cd00;\">     {<\/span>\n<span style=\"color: #00cd00;\">       prev_logic=\\$0;<\/span>\n<span style=\"color: #00cd00;\">     } \"<\/span> | feedgnuplot --stream --ylabel <span style=\"color: #00cd00;\">'Period (ms)'<\/span> --equation <span style=\"color: #00cd00;\">\"1000.\/$pulserate title \\\"True ${pulserate}Hz period\\\"\"<\/span>\n<\/pre>\n<\/div>\n\n<p>\nOn the server I was using (physical RS-232 port, ancient 3.something kernel):\n<\/p>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2025\/03\/14_getting-precise-timings-out-of-rs-232-output\/timings.hw.serial-server.svg\" alt=\"timings.hw.serial-server.svg\" width=\"80%\" \/>\n<\/p>\n<\/div>\n\n<p>\nOK&#x2026; This is very discrete for some reason, and generally worse than 0.1ms.\nWhat about my laptop (physical RS-232 port, recent 6.12 kernel)?\n<\/p>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2025\/03\/14_getting-precise-timings-out-of-rs-232-output\/timings.hw.serial-laptop.svg\" alt=\"timings.hw.serial-laptop.svg\" width=\"80%\" \/>\n<\/p>\n<\/div>\n\n<p>\nNot discrete anymore, but not really any more precise. What about using a\nusb-serial converter? I expect this to be worse.\n<\/p>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2025\/03\/14_getting-precise-timings-out-of-rs-232-output\/timings.hw.usbserial.svg\" alt=\"timings.hw.usbserial.svg\" width=\"80%\" \/>\n<\/p>\n<\/div>\n\n<p>\nYeah, looks worse. For my purposes, an accuracy of 0.1ms is marginal, and the\nhardware adds non-negligible errors. So I cut my losses, and use an external\nsignal generator:\n<\/p>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2025\/03\/14_getting-precise-timings-out-of-rs-232-output\/timings.hw.generator.svg\" alt=\"timings.hw.generator.svg\" width=\"80%\" \/>\n<\/p>\n<\/div>\n\n<p>\nYeah. That's better, so that's what I use.\n<\/p>\n"},{"title":"Shop scheduling with PuLP","author":{"name":"Dima Kogan"},"link":{"@attributes":{"href":"http:\/\/notes.secretsauce.net\/notes\/2025\/03\/05_shop-scheduling-with-pulp.html"}},"updated":"2025-03-05T12:02:00Z","published":"2025-03-05T12:02:00Z","id":"notes\/2025\/03\/05_shop-scheduling-with-pulp.html","category":[{"@attributes":{"scheme":"\/tags\/data.html","term":"data","label":"data"}},{"@attributes":{"scheme":"\/tags\/tools.html","term":"tools","label":"tools"}}],"content":"<p>\nI recently used the <a href=\"https:\/\/coin-or.github.io\/pulp\/\">PuLP modeler<\/a> to solve a work scheduling problem to assign\nworkers to shifts. Here are notes about doing that. This is a common use case,\nbut isn't explicitly covered in the <a href=\"https:\/\/coin-or.github.io\/pulp\/CaseStudies\/index.html\">case studies<\/a> in the PuLP documentation.\n<\/p>\n\n<p>\nHere's the problem:\n<\/p>\n\n<ul class=\"org-ul\">\n<li>We are trying to put together a schedule for one week\n<\/li>\n<li>Each day has some set of work shifts that need to be staffed\n<\/li>\n<li>Each shift must be staffed with <i>exactly<\/i> one worker\n<\/li>\n<li>The shift schedule is known beforehand, and the workers each declare their\npreferences beforehand: they mark each shift in the week as one of:\n<ul class=\"org-ul\">\n<li>PREFERRED (if they want to be scheduled on that shift)\n<\/li>\n<li>NEUTRAL\n<\/li>\n<li>DISFAVORED (if they don't love that shift)\n<\/li>\n<li>REFUSED (if they absolutely cannot work that shift)\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n<p>\nThe tool is supposed to allocate workers to the shifts to try to cover all the\nshifts, give everybody work, and try to match their preferences. I implemented\nthe tool:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-python\"><span style=\"color: #cdcd00;\">#<\/span><span style=\"color: #cdcd00;\">!\/usr\/bin\/python3<\/span>\n\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> sys\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> os\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> re\n\n<span style=\"color: #00cdcd; font-weight: bold;\">def<\/span> <span style=\"color: #0000ee; font-weight: bold;\">report_solution_to_console<\/span>(<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>):\n    <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> w <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> days_of_week:\n        <span style=\"color: #cdcd00;\">annotation<\/span> = <span style=\"color: #00cd00;\">''<\/span>\n        <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> human_annotate <span style=\"color: #00cdcd; font-weight: bold;\">is<\/span> <span style=\"color: #00cdcd; font-weight: bold;\">not<\/span> <span style=\"color: #cd00cd;\">None<\/span>:\n            <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys():\n                <span style=\"color: #cdcd00;\">m<\/span> = re.match(rf<span style=\"color: #00cd00;\">'{w} - '<\/span>, s)\n                <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> <span style=\"color: #00cdcd; font-weight: bold;\">not<\/span> m: <span style=\"color: #00cdcd; font-weight: bold;\">continue<\/span>\n                <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> <span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[human_annotate][s].value():\n                    <span style=\"color: #cdcd00;\">annotation<\/span> = f<span style=\"color: #00cd00;\">\" ({human_annotate} SCHEDULED)\"<\/span>\n                    <span style=\"color: #00cdcd; font-weight: bold;\">break<\/span>\n            <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> <span style=\"color: #00cdcd; font-weight: bold;\">not<\/span> <span style=\"color: #0000ee; font-weight: bold;\">len<\/span>(annotation):\n                <span style=\"color: #cdcd00;\">annotation<\/span> = f<span style=\"color: #00cd00;\">\" ({human_annotate} OFF)\"<\/span>\n\n        <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">\"{w}{annotation}\"<\/span>)\n\n        <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys():\n            <span style=\"color: #cdcd00;\">m<\/span> = re.match(rf<span style=\"color: #00cd00;\">'{w} - '<\/span>, s)\n            <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> <span style=\"color: #00cdcd; font-weight: bold;\">not<\/span> m: <span style=\"color: #00cdcd; font-weight: bold;\">continue<\/span>\n\n            <span style=\"color: #cdcd00;\">annotation<\/span> = <span style=\"color: #00cd00;\">''<\/span>\n            <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> human_annotate <span style=\"color: #00cdcd; font-weight: bold;\">is<\/span> <span style=\"color: #00cdcd; font-weight: bold;\">not<\/span> <span style=\"color: #cd00cd;\">None<\/span>:\n                <span style=\"color: #cdcd00;\">annotation<\/span> = f<span style=\"color: #00cd00;\">\" ({human_annotate} {shifts[s][human_annotate]})\"<\/span>\n            <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">\"    ---- {s[m.end():]}{annotation}\"<\/span>)\n\n            <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans:\n                <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> <span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s].value():\n                    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">\"         {h} ({shifts[s][h]})\"<\/span>)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">def<\/span> <span style=\"color: #0000ee; font-weight: bold;\">report_solution_summary_to_console<\/span>(<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>):\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"\\nSUMMARY\"<\/span>)\n\n    <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans:\n        <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">\"-- {h}\"<\/span>)\n        <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">\"   benefit: {benefits[h].value():.3f}\"<\/span>)\n\n        <span style=\"color: #cdcd00;\">counts<\/span> = <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>()\n        <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> a <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> availabilities:\n            <span style=\"color: #cdcd00;\">counts<\/span>[a] = 0\n\n        <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys():\n            <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> <span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s].value():\n                counts[shifts[s][h]] += 1\n\n        <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> a <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> availabilities:\n            <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">\"   {counts[a]} {a}\"<\/span>)\n\n\n<span style=\"color: #cdcd00;\">human_annotate<\/span> = <span style=\"color: #cd00cd;\">None<\/span>\n\n<span style=\"color: #cdcd00;\">days_of_week<\/span> = (<span style=\"color: #00cd00;\">'SUNDAY'<\/span>,\n                <span style=\"color: #00cd00;\">'MONDAY'<\/span>,\n                <span style=\"color: #00cd00;\">'TUESDAY'<\/span>,\n                <span style=\"color: #00cd00;\">'WEDNESDAY'<\/span>,\n                <span style=\"color: #00cd00;\">'THURSDAY'<\/span>,\n                <span style=\"color: #00cd00;\">'FRIDAY'<\/span>,\n                <span style=\"color: #00cd00;\">'SATURDAY'<\/span>)\n\n<span style=\"color: #cdcd00;\">humans<\/span> = [<span style=\"color: #00cd00;\">'ALICE'<\/span>, <span style=\"color: #00cd00;\">'BOB'<\/span>,\n          <span style=\"color: #00cd00;\">'CAROL'<\/span>, <span style=\"color: #00cd00;\">'DAVID'<\/span>, <span style=\"color: #00cd00;\">'EVE'<\/span>, <span style=\"color: #00cd00;\">'FRANK'<\/span>, <span style=\"color: #00cd00;\">'GRACE'<\/span>, <span style=\"color: #00cd00;\">'HEIDI'<\/span>, <span style=\"color: #00cd00;\">'IVAN'<\/span>, <span style=\"color: #00cd00;\">'JUDY'<\/span>]\n\n<span style=\"color: #cdcd00;\">shifts<\/span> = {<span style=\"color: #00cd00;\">'SUNDAY - SANDING 9:00 AM - 4:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>},\n          <span style=\"color: #00cd00;\">'WEDNESDAY - SAWING 7:30 AM - 2:30 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'THURSDAY - SANDING 9:00 AM - 4:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>},\n          <span style=\"color: #00cd00;\">'SATURDAY - SAWING 7:30 AM - 2:30 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'SUNDAY - SAWING 9:00 AM - 4:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'MONDAY - SAWING 9:00 AM - 4:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'TUESDAY - SAWING 9:00 AM - 4:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'WEDNESDAY - PAINTING 7:30 AM - 2:30 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'THURSDAY - SAWING 9:00 AM - 4:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'FRIDAY - SAWING 9:00 AM - 4:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'SATURDAY - PAINTING 7:30 AM - 2:30 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'SUNDAY - PAINTING 9:45 AM - 4:45 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'MONDAY - PAINTING 9:45 AM - 4:45 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'TUESDAY - PAINTING 9:45 AM - 4:45 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'WEDNESDAY - SANDING 9:45 AM - 4:45 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'THURSDAY - PAINTING 9:45 AM - 4:45 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'FRIDAY - PAINTING 9:45 AM - 4:45 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'SATURDAY - SANDING 9:45 AM - 4:45 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'SUNDAY - PAINTING 11:00 AM - 6:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'MONDAY - PAINTING 12:00 PM - 7:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'TUESDAY - PAINTING 12:00 PM - 7:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'WEDNESDAY - PAINTING 12:00 PM - 7:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'THURSDAY - PAINTING 12:00 PM - 7:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'FRIDAY - PAINTING 12:00 PM - 7:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'SATURDAY - PAINTING 12:00 PM - 7:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'SUNDAY - SAWING 12:00 PM - 7:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'MONDAY - SAWING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'TUESDAY - SAWING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'WEDNESDAY - SAWING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'THURSDAY - SAWING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'FRIDAY - SAWING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'SATURDAY - SAWING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'SUNDAY - PAINTING 12:15 PM - 7:15 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'PREFERRED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'MONDAY - PAINTING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'TUESDAY - PAINTING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'WEDNESDAY - PAINTING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'THURSDAY - PAINTING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'FRIDAY - PAINTING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>},\n          <span style=\"color: #00cd00;\">'SATURDAY - PAINTING 2:00 PM - 9:00 PM'<\/span>:\n          {<span style=\"color: #00cd00;\">'ALICE'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'BOB'<\/span>:   <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'CAROL'<\/span>: <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'FRANK'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'HEIDI'<\/span>: <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>,\n           <span style=\"color: #00cd00;\">'IVAN'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'JUDY'<\/span>:  <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>,\n           <span style=\"color: #00cd00;\">'EVE'<\/span>:   <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'GRACE'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>,\n           <span style=\"color: #00cd00;\">'DAVID'<\/span>: <span style=\"color: #00cd00;\">'REFUSED'<\/span>}}\n\n<span style=\"color: #cdcd00;\">availabilities<\/span> = [<span style=\"color: #00cd00;\">'PREFERRED'<\/span>, <span style=\"color: #00cd00;\">'NEUTRAL'<\/span>, <span style=\"color: #00cd00;\">'DISFAVORED'<\/span>]\n\n\n\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> pulp\n<span style=\"color: #cdcd00;\">prob<\/span> = pulp.LpProblem(<span style=\"color: #00cd00;\">\"Scheduling\"<\/span>, pulp.LpMaximize)\n\n<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span> = pulp.LpVariable.dicts(<span style=\"color: #00cd00;\">\"Assignments\"<\/span>,\n                             (humans, shifts.keys()),\n                             <span style=\"color: #cd00cd;\">None<\/span>,<span style=\"color: #cd00cd;\">None<\/span>, <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">bounds; unused, since these are binary variables<\/span>\n                             pulp.LpBinary)\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">Everyone works at least 2 shifts<\/span>\n<span style=\"color: #cdcd00;\">Nshifts_min<\/span> = 2\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans:\n    <span style=\"color: #cdcd00;\">prob<\/span> += (\n        pulp.lpSum([<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s] <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys()]) &gt;= Nshifts_min,\n        f<span style=\"color: #00cd00;\">\"{h} works at least {Nshifts_min} shifts\"<\/span>,\n    )\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">each shift is ~ 8 hours, so I limit everyone to 40\/8 = 5 shifts<\/span>\n<span style=\"color: #cdcd00;\">Nshifts_max<\/span> = 5\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans:\n    <span style=\"color: #cdcd00;\">prob<\/span> += (\n        pulp.lpSum([<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s] <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys()]) &lt;= Nshifts_max,\n        f<span style=\"color: #00cd00;\">\"{h} works at most {Nshifts_max} shifts\"<\/span>,\n    )\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">all shifts staffed and not double-staffed<\/span>\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys():\n    <span style=\"color: #cdcd00;\">prob<\/span> += (\n        pulp.lpSum([<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s] <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans]) == 1,\n        f<span style=\"color: #00cd00;\">\"{s} is staffed\"<\/span>,\n    )\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">each human can work at most one shift on any given day<\/span>\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> w <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> days_of_week:\n    <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans:\n        <span style=\"color: #cdcd00;\">prob<\/span> += (\n            pulp.lpSum([<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s] <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys() <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> re.match(rf<span style=\"color: #00cd00;\">'{w} '<\/span>,s)]) &lt;= 1,\n            f<span style=\"color: #00cd00;\">\"{h} cannot be double-booked on {w}\"<\/span>\n        )\n\n\n<span style=\"color: #cdcd00;\">#### <\/span><span style=\"color: #cdcd00;\">Some explicit constraints; as an example<\/span>\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">DAVID can't work any PAINTING shift and is off on Thu and Sun<\/span>\n<span style=\"color: #cdcd00;\">h<\/span> = <span style=\"color: #00cd00;\">'DAVID'<\/span>\n<span style=\"color: #cdcd00;\">prob<\/span> += (\n    pulp.lpSum([<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s] <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys() <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> re.search(r<span style=\"color: #00cd00;\">'- PAINTING'<\/span>,s)]) == 0,\n    f<span style=\"color: #00cd00;\">\"{h} can't work any PAINTING shift\"<\/span>\n)\n<span style=\"color: #cdcd00;\">prob<\/span> += (\n    pulp.lpSum([<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s] <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys() <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> re.match(r<span style=\"color: #00cd00;\">'THURSDAY|SUNDAY'<\/span>,s)]) == 0,\n    f<span style=\"color: #00cd00;\">\"{h} is off on Thursday and Sunday\"<\/span>\n)\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">Do not assign any \"REFUSED\" shifts<\/span>\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys():\n    <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans:\n        <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> shifts[s][h] == <span style=\"color: #00cd00;\">'REFUSED'<\/span>:\n            <span style=\"color: #cdcd00;\">prob<\/span> += (\n                <span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s] == 0,\n                f<span style=\"color: #00cd00;\">\"{h} is not available for {s}\"<\/span>\n            )\n\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">Objective. I try to maximize the \"happiness\". Each human sees each shift as<\/span>\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">one of:<\/span>\n<span style=\"color: #cdcd00;\">#<\/span>\n<span style=\"color: #cdcd00;\">#   <\/span><span style=\"color: #cdcd00;\">PREFERRED<\/span>\n<span style=\"color: #cdcd00;\">#   <\/span><span style=\"color: #cdcd00;\">NEUTRAL<\/span>\n<span style=\"color: #cdcd00;\">#   <\/span><span style=\"color: #cdcd00;\">DISFAVORED<\/span>\n<span style=\"color: #cdcd00;\">#   <\/span><span style=\"color: #cdcd00;\">REFUSED<\/span>\n<span style=\"color: #cdcd00;\">#<\/span>\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">I set a hard constraint to handle \"REFUSED\", and arbitrarily, I set these<\/span>\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">benefit values for the others<\/span>\n<span style=\"color: #cdcd00;\">benefit_availability<\/span> = <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>()\n<span style=\"color: #cdcd00;\">benefit_availability<\/span>[<span style=\"color: #00cd00;\">'PREFERRED'<\/span>]  = 3\n<span style=\"color: #cdcd00;\">benefit_availability<\/span>[<span style=\"color: #00cd00;\">'NEUTRAL'<\/span>]    = 2\n<span style=\"color: #cdcd00;\">benefit_availability<\/span>[<span style=\"color: #00cd00;\">'DISFAVORED'<\/span>] = 1\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">Not used, since this is a hard constraint. But the code needs this to be a<\/span>\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">part of the benefit. I can ignore these in the code, but let's keep this<\/span>\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">simple<\/span>\n<span style=\"color: #cdcd00;\">benefit_availability<\/span>[<span style=\"color: #00cd00;\">'REFUSED'<\/span> ] = -1000\n\n<span style=\"color: #cdcd00;\">benefits<\/span> = <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>()\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans:\n    <span style=\"color: #cdcd00;\">benefits<\/span>[h] = \\\n        pulp.lpSum([<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s] * benefit_availability[shifts[s][h]] \\\n                    <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys()])\n\n<span style=\"color: #cdcd00;\">benefit_total<\/span> = \\\n    pulp.lpSum([benefits[h] \\\n                <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans])\n\n<span style=\"color: #cdcd00;\">prob<\/span> += (\n    benefit_total,\n    <span style=\"color: #00cd00;\">\"happiness\"<\/span>,\n)\n\nprob.solve()\n\n<span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> pulp.LpStatus[prob.status] == <span style=\"color: #00cd00;\">\"Optimal\"<\/span>:\n    report_solution_to_console(<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>)\n    report_solution_summary_to_console(<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>)\n<\/pre>\n<\/div>\n\n<p>\nThe set of workers is in the <code>humans<\/code> variable, and the shift schedule and the\nworkers' preferences are encoded in the <code>shifts<\/code> dict. The problem is defined by\na <code>vars<\/code> dict of dicts, each a boolean variable indicating whether a particular\nworker is scheduled for a particular shift. We define a set of constraints to\nthese worker allocations to restrict ourselves to <i>valid<\/i> solutions. And among\nthese valid solutions, we try to find the one that maximizes some <i>benefit<\/i>\nfunction, defined here as:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-python\"><span style=\"color: #cdcd00;\">benefit_availability<\/span> = <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>()\n<span style=\"color: #cdcd00;\">benefit_availability<\/span>[<span style=\"color: #00cd00;\">'PREFERRED'<\/span>]  = 3\n<span style=\"color: #cdcd00;\">benefit_availability<\/span>[<span style=\"color: #00cd00;\">'NEUTRAL'<\/span>]    = 2\n<span style=\"color: #cdcd00;\">benefit_availability<\/span>[<span style=\"color: #00cd00;\">'DISFAVORED'<\/span>] = 1\n\n<span style=\"color: #cdcd00;\">benefits<\/span> = <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>()\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans:\n    <span style=\"color: #cdcd00;\">benefits<\/span>[h] = \\\n        pulp.lpSum([<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s] * benefit_availability[shifts[s][h]] \\\n                    <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys()])\n\n<span style=\"color: #cdcd00;\">benefit_total<\/span> = \\\n    pulp.lpSum([benefits[h] \\\n                <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans])\n<\/pre>\n<\/div>\n\n<p>\nSo for instance each shift that was scheduled as somebody's PREFERRED shift\ngives us 3 benefit points. And if all the shifts ended up being PREFERRED, we'd\nhave a total benefit value of 3*Nshifts. This is impossible, however, because\nthat would violate some constraints in the problem.\n<\/p>\n\n<p>\nThe exact trade-off between the different preferences is set in the\n<code>benefit_availability<\/code> dict. With the above numbers, it's equally good for\nsomebody to have a NEUTRAL shift and a day off as it is for them to have\nDISFAVORED shifts. If we really want to encourage the program to work people as\nmuch as possible (days off discouraged), we'd want to raise the DISFAVORED\nthreshold.\n<\/p>\n\n<p>\nI run this program and I get:\n<\/p>\n\n<pre class=\"example\">\n....\nResult - Optimal solution found\n\nObjective value:                108.00000000\nEnumerated nodes:               0\nTotal iterations:               0\nTime (CPU seconds):             0.01\nTime (Wallclock seconds):       0.01\n\nOption for printingOptions changed from normal to all\nTotal time (CPU seconds):       0.02   (Wallclock seconds):       0.02\n\nSUNDAY\n    ---- SANDING 9:00 AM - 4:00 PM\n         EVE (PREFERRED)\n    ---- SAWING 9:00 AM - 4:00 PM\n         IVAN (PREFERRED)\n    ---- PAINTING 9:45 AM - 4:45 PM\n         FRANK (PREFERRED)\n    ---- PAINTING 11:00 AM - 6:00 PM\n         HEIDI (PREFERRED)\n    ---- SAWING 12:00 PM - 7:00 PM\n         ALICE (PREFERRED)\n    ---- PAINTING 12:15 PM - 7:15 PM\n         CAROL (PREFERRED)\nMONDAY\n    ---- SAWING 9:00 AM - 4:00 PM\n         DAVID (PREFERRED)\n    ---- PAINTING 9:45 AM - 4:45 PM\n         IVAN (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM\n         GRACE (PREFERRED)\n    ---- SAWING 2:00 PM - 9:00 PM\n         ALICE (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM\n         HEIDI (NEUTRAL)\nTUESDAY\n    ---- SAWING 9:00 AM - 4:00 PM\n         DAVID (PREFERRED)\n    ---- PAINTING 9:45 AM - 4:45 PM\n         EVE (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM\n         FRANK (NEUTRAL)\n    ---- SAWING 2:00 PM - 9:00 PM\n         BOB (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM\n         HEIDI (NEUTRAL)\nWEDNESDAY\n    ---- SAWING 7:30 AM - 2:30 PM\n         DAVID (PREFERRED)\n    ---- PAINTING 7:30 AM - 2:30 PM\n         IVAN (PREFERRED)\n    ---- SANDING 9:45 AM - 4:45 PM\n         FRANK (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM\n         JUDY (PREFERRED)\n    ---- SAWING 2:00 PM - 9:00 PM\n         BOB (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM\n         ALICE (NEUTRAL)\nTHURSDAY\n    ---- SANDING 9:00 AM - 4:00 PM\n         GRACE (PREFERRED)\n    ---- SAWING 9:00 AM - 4:00 PM\n         CAROL (PREFERRED)\n    ---- PAINTING 9:45 AM - 4:45 PM\n         EVE (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM\n         JUDY (PREFERRED)\n    ---- SAWING 2:00 PM - 9:00 PM\n         BOB (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM\n         ALICE (NEUTRAL)\nFRIDAY\n    ---- SAWING 9:00 AM - 4:00 PM\n         DAVID (PREFERRED)\n    ---- PAINTING 9:45 AM - 4:45 PM\n         FRANK (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM\n         GRACE (NEUTRAL)\n    ---- SAWING 2:00 PM - 9:00 PM\n         BOB (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM\n         HEIDI (NEUTRAL)\nSATURDAY\n    ---- SAWING 7:30 AM - 2:30 PM\n         CAROL (PREFERRED)\n    ---- PAINTING 7:30 AM - 2:30 PM\n         IVAN (PREFERRED)\n    ---- SANDING 9:45 AM - 4:45 PM\n         DAVID (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM\n         FRANK (NEUTRAL)\n    ---- SAWING 2:00 PM - 9:00 PM\n         ALICE (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM\n         BOB (NEUTRAL)\n\nSUMMARY\n-- ALICE\n   benefit: 13.000\n   3 PREFERRED\n   2 NEUTRAL\n   0 DISFAVORED\n-- BOB\n   benefit: 14.000\n   4 PREFERRED\n   1 NEUTRAL\n   0 DISFAVORED\n-- CAROL\n   benefit: 9.000\n   3 PREFERRED\n   0 NEUTRAL\n   0 DISFAVORED\n-- DAVID\n   benefit: 15.000\n   5 PREFERRED\n   0 NEUTRAL\n   0 DISFAVORED\n-- EVE\n   benefit: 9.000\n   3 PREFERRED\n   0 NEUTRAL\n   0 DISFAVORED\n-- FRANK\n   benefit: 13.000\n   3 PREFERRED\n   2 NEUTRAL\n   0 DISFAVORED\n-- GRACE\n   benefit: 8.000\n   2 PREFERRED\n   1 NEUTRAL\n   0 DISFAVORED\n-- HEIDI\n   benefit: 9.000\n   1 PREFERRED\n   3 NEUTRAL\n   0 DISFAVORED\n-- IVAN\n   benefit: 12.000\n   4 PREFERRED\n   0 NEUTRAL\n   0 DISFAVORED\n-- JUDY\n   benefit: 6.000\n   2 PREFERRED\n   0 NEUTRAL\n   0 DISFAVORED\n<\/pre>\n\n<p>\nSo we have a solution! We have 108 total benefit points. But it looks a bit\nuneven: Judy only works 2 days, while some people work many more: David works 5\nfor instance. Why is that? I update the program with =human_annotate = 'JUDY'=,\nrun it again, and it tells me more about Judy's preferences:\n<\/p>\n\n<pre class=\"example\">\nObjective value:                108.00000000\nEnumerated nodes:               0\nTotal iterations:               0\nTime (CPU seconds):             0.01\nTime (Wallclock seconds):       0.01\n\nOption for printingOptions changed from normal to all\nTotal time (CPU seconds):       0.01   (Wallclock seconds):       0.02\n\nSUNDAY (JUDY OFF)\n    ---- SANDING 9:00 AM - 4:00 PM (JUDY NEUTRAL)\n         EVE (PREFERRED)\n    ---- SAWING 9:00 AM - 4:00 PM (JUDY PREFERRED)\n         IVAN (PREFERRED)\n    ---- PAINTING 9:45 AM - 4:45 PM (JUDY PREFERRED)\n         FRANK (PREFERRED)\n    ---- PAINTING 11:00 AM - 6:00 PM (JUDY NEUTRAL)\n         HEIDI (PREFERRED)\n    ---- SAWING 12:00 PM - 7:00 PM (JUDY PREFERRED)\n         ALICE (PREFERRED)\n    ---- PAINTING 12:15 PM - 7:15 PM (JUDY NEUTRAL)\n         CAROL (PREFERRED)\nMONDAY (JUDY OFF)\n    ---- SAWING 9:00 AM - 4:00 PM (JUDY PREFERRED)\n         DAVID (PREFERRED)\n    ---- PAINTING 9:45 AM - 4:45 PM (JUDY NEUTRAL)\n         IVAN (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM (JUDY NEUTRAL)\n         GRACE (PREFERRED)\n    ---- SAWING 2:00 PM - 9:00 PM (JUDY DISFAVORED)\n         ALICE (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM (JUDY DISFAVORED)\n         HEIDI (NEUTRAL)\nTUESDAY (JUDY OFF)\n    ---- SAWING 9:00 AM - 4:00 PM (JUDY PREFERRED)\n         DAVID (PREFERRED)\n    ---- PAINTING 9:45 AM - 4:45 PM (JUDY PREFERRED)\n         EVE (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM (JUDY REFUSED)\n         FRANK (NEUTRAL)\n    ---- SAWING 2:00 PM - 9:00 PM (JUDY REFUSED)\n         BOB (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM (JUDY REFUSED)\n         HEIDI (NEUTRAL)\nWEDNESDAY (JUDY SCHEDULED)\n    ---- SAWING 7:30 AM - 2:30 PM (JUDY REFUSED)\n         DAVID (PREFERRED)\n    ---- PAINTING 7:30 AM - 2:30 PM (JUDY REFUSED)\n         IVAN (PREFERRED)\n    ---- SANDING 9:45 AM - 4:45 PM (JUDY NEUTRAL)\n         FRANK (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM (JUDY PREFERRED)\n         JUDY (PREFERRED)\n    ---- SAWING 2:00 PM - 9:00 PM (JUDY DISFAVORED)\n         BOB (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM (JUDY DISFAVORED)\n         ALICE (NEUTRAL)\nTHURSDAY (JUDY SCHEDULED)\n    ---- SANDING 9:00 AM - 4:00 PM (JUDY PREFERRED)\n         GRACE (PREFERRED)\n    ---- SAWING 9:00 AM - 4:00 PM (JUDY PREFERRED)\n         CAROL (PREFERRED)\n    ---- PAINTING 9:45 AM - 4:45 PM (JUDY PREFERRED)\n         EVE (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM (JUDY PREFERRED)\n         JUDY (PREFERRED)\n    ---- SAWING 2:00 PM - 9:00 PM (JUDY DISFAVORED)\n         BOB (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM (JUDY DISFAVORED)\n         ALICE (NEUTRAL)\nFRIDAY (JUDY OFF)\n    ---- SAWING 9:00 AM - 4:00 PM (JUDY DISFAVORED)\n         DAVID (PREFERRED)\n    ---- PAINTING 9:45 AM - 4:45 PM (JUDY DISFAVORED)\n         FRANK (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM (JUDY DISFAVORED)\n         GRACE (NEUTRAL)\n    ---- SAWING 2:00 PM - 9:00 PM (JUDY REFUSED)\n         BOB (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM (JUDY REFUSED)\n         HEIDI (NEUTRAL)\nSATURDAY (JUDY OFF)\n    ---- SAWING 7:30 AM - 2:30 PM (JUDY REFUSED)\n         CAROL (PREFERRED)\n    ---- PAINTING 7:30 AM - 2:30 PM (JUDY REFUSED)\n         IVAN (PREFERRED)\n    ---- SANDING 9:45 AM - 4:45 PM (JUDY REFUSED)\n         DAVID (PREFERRED)\n    ---- PAINTING 12:00 PM - 7:00 PM (JUDY DISFAVORED)\n         FRANK (NEUTRAL)\n    ---- SAWING 2:00 PM - 9:00 PM (JUDY DISFAVORED)\n         ALICE (PREFERRED)\n    ---- PAINTING 2:00 PM - 9:00 PM (JUDY DISFAVORED)\n         BOB (NEUTRAL)\n\nSUMMARY\n-- ALICE\n   benefit: 13.000\n   3 PREFERRED\n   2 NEUTRAL\n   0 DISFAVORED\n-- BOB\n   benefit: 14.000\n   4 PREFERRED\n   1 NEUTRAL\n   0 DISFAVORED\n-- CAROL\n   benefit: 9.000\n   3 PREFERRED\n   0 NEUTRAL\n   0 DISFAVORED\n-- DAVID\n   benefit: 15.000\n   5 PREFERRED\n   0 NEUTRAL\n   0 DISFAVORED\n-- EVE\n   benefit: 9.000\n   3 PREFERRED\n   0 NEUTRAL\n   0 DISFAVORED\n-- FRANK\n   benefit: 13.000\n   3 PREFERRED\n   2 NEUTRAL\n   0 DISFAVORED\n-- GRACE\n   benefit: 8.000\n   2 PREFERRED\n   1 NEUTRAL\n   0 DISFAVORED\n-- HEIDI\n   benefit: 9.000\n   1 PREFERRED\n   3 NEUTRAL\n   0 DISFAVORED\n-- IVAN\n   benefit: 12.000\n   4 PREFERRED\n   0 NEUTRAL\n   0 DISFAVORED\n-- JUDY\n   benefit: 6.000\n   2 PREFERRED\n   0 NEUTRAL\n   0 DISFAVORED\n<\/pre>\n\n<p>\nThis tells us that on Monday Judy does not work, although she marked the SAWING\nshift as PREFERRED. Instead David got that shift. What would happen if David\ngave that shift to Judy? He would lose 3 points, she would gain 3 points, and\nthe total would remain exactly the same at 108.\n<\/p>\n\n<p>\nHow would we favor a more even distribution? We need some sort of tie-break. I\nwant to add a nonlinearity to strongly disfavor people getting a low number of\nshifts. But PuLP is very explicitly a <i>linear<\/i> programming solver, and cannot\nsolve nonlinear problems. Here we can get around this by enumerating each\nspecific case, and assigning it a nonlinear benefit function. The most obvious\napproach is to define another set of boolean variables:\n<code>vars_Nshifts[human][N]<\/code>. And then using them to add extra benefit terms, with\nvalues nonlinearly related to <code>Nshifts<\/code>. Something like this:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-python\"><span style=\"color: #cdcd00;\">benefit_boost_Nshifts<\/span> = \\\n    {2: -0.8,\n     3: -0.5,\n     4: -0.3,\n     5: -0.2}\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans:\n    <span style=\"color: #cdcd00;\">benefits<\/span>[h] = \\\n        ... + \\\n        pulp.lpSum([vars_Nshifts[h][n] * benefit_boost_Nshifts[n] \\\n                    <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> n <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> benefit_boost_Nshifts.keys()])\n<\/pre>\n<\/div>\n\n<p>\nSo in the previous example we considered giving David's 5th shift to Judy, for\nher 3rd shift. In that scenario, David's extra benefit would change from -0.2 to\n-0.3 (a shift of -0.1), while Judy's would change from -0.8 to -0.5 (a shift of\n+0.3). So the balancing out the shifts in this way would work: the solver would\nfavor the solution with the higher benefit function.\n<\/p>\n\n<p>\nGreat. In order for this to work, we need the <code>vars_Nshifts[human][N]<\/code> variables\nto function as intended: they need to be binary indicators of whether a specific\nperson has that many shifts or not. That would need to be implemented with\nconstraints. Let's plot it like this:\n<\/p>\n\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-python\"><span style=\"color: #cdcd00;\">#<\/span><span style=\"color: #cdcd00;\">!\/usr\/bin\/python3<\/span>\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> numpy <span style=\"color: #00cdcd; font-weight: bold;\">as<\/span> np\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> gnuplotlib <span style=\"color: #00cdcd; font-weight: bold;\">as<\/span> gp\n\n<span style=\"color: #cdcd00;\">Nshifts_eq<\/span>  = 4\n<span style=\"color: #cdcd00;\">Nshifts_max<\/span> = 10\n\n<span style=\"color: #cdcd00;\">Nshifts<\/span> = np.arange(Nshifts_max+1)\n<span style=\"color: #cdcd00;\">i0<\/span> = np.nonzero(Nshifts != Nshifts_eq)[0]\n<span style=\"color: #cdcd00;\">i1<\/span> = np.nonzero(Nshifts == Nshifts_eq)[0]\n\ngp.plot( <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">True value: var_Nshifts4==0, Nshifts!=4<\/span>\n         ( np.zeros(i0.shape),\n           Nshifts[i0],\n           <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>(_with     = <span style=\"color: #00cd00;\">'points pt 7 ps 1 lc \"red\"'<\/span>) ),\n         <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">True value: var_Nshifts4==1, Nshifts==4<\/span>\n         ( np.ones(i1.shape),\n           Nshifts[i1],\n           <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>(_with     = <span style=\"color: #00cd00;\">'points pt 7 ps 1 lc \"red\"'<\/span>) ),\n         <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">False value: var_Nshifts4==1, Nshifts!=4<\/span>\n         ( np.ones(i0.shape),\n           Nshifts[i0],\n           <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>(_with     = <span style=\"color: #00cd00;\">'points pt 7 ps 1 lc \"black\"'<\/span>) ),\n         <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">False value: var_Nshifts4==0, Nshifts==4<\/span>\n         ( np.zeros(i1.shape),\n           Nshifts[i1],\n           <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>(_with     = <span style=\"color: #00cd00;\">'points pt 7 ps 1 lc \"black\"'<\/span>) ),\n        unset=(<span style=\"color: #00cd00;\">'grid'<\/span>),\n        _set = (f<span style=\"color: #00cd00;\">'xtics (\"(Nshifts=={Nshifts_eq}) == 0\" 0, \"(Nshifts=={Nshifts_eq}) == 1\" 1)'<\/span>),\n        _xrange = (-0.1, 1.1),\n        ylabel = <span style=\"color: #00cd00;\">\"Nshifts\"<\/span>,\n        title = <span style=\"color: #00cd00;\">\"Nshifts equality variable: not linearly separable\"<\/span>,\n        hardcopy = <span style=\"color: #00cd00;\">\"\/tmp\/scheduling-Nshifts-eq.svg\"<\/span>)\n<\/pre>\n<\/div>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2025\/03\/05_shop-scheduling-with-pulp\/scheduling-Nshifts-eq.svg\" alt=\"scheduling-Nshifts-eq.svg\" width=\"80%\" \/>\n<\/p>\n<\/div>\n\n<p>\nSo a hypothetical <code>vars_Nshifts[h][4]<\/code> variable (plotted on the x axis of this\nplot) would need to be defined by a set of linear AND constraints to <a href=\"https:\/\/en.wikipedia.org\/wiki\/Linear_separability\">linearly\nseparate<\/a> the true (red) values of this variable from the false (black) values.\nAs can be seen in this plot, this isn't possible. So this representation does\n<i>not<\/i> work.\n<\/p>\n\n<p>\nHow do we fix it? We can use inequality variables instead. I define a different\nset of variables <code>vars_Nshifts_leq[human][N]<\/code> that are 1 iff <code>Nshifts<\/code> &lt;= <code>N<\/code>.\nThe equality variable from before can be expressed as a difference of these\ninequality variables: <code>vars_Nshifts[human][N] =\nvars_Nshifts_leq[human][N]-vars_Nshifts_leq[human][N-1]<\/code>\n<\/p>\n\n<p>\nCan these <code>vars_Nshifts_leq<\/code> variables be defined by a set of linear AND\nconstraints? Yes:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-python\"><span style=\"color: #cdcd00;\">#<\/span><span style=\"color: #cdcd00;\">!\/usr\/bin\/python3<\/span>\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> numpy <span style=\"color: #00cdcd; font-weight: bold;\">as<\/span> np\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> numpysane <span style=\"color: #00cdcd; font-weight: bold;\">as<\/span> nps\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> gnuplotlib <span style=\"color: #00cdcd; font-weight: bold;\">as<\/span> gp\n\n<span style=\"color: #cdcd00;\">Nshifts_leq<\/span> = 4\n<span style=\"color: #cdcd00;\">Nshifts_max<\/span> = 10\n\n<span style=\"color: #cdcd00;\">Nshifts<\/span> = np.arange(Nshifts_max+1)\n<span style=\"color: #cdcd00;\">i0<\/span> = np.nonzero(Nshifts &gt;  Nshifts_leq)[0]\n<span style=\"color: #cdcd00;\">i1<\/span> = np.nonzero(Nshifts &lt;= Nshifts_leq)[0]\n\n<span style=\"color: #00cdcd; font-weight: bold;\">def<\/span> <span style=\"color: #0000ee; font-weight: bold;\">linear_slope_yintercept<\/span>(xy0,xy1):\n    <span style=\"color: #cdcd00;\">m<\/span> = (xy1[1] - xy0[1])\/(xy1[0] - xy0[0])\n    <span style=\"color: #cdcd00;\">b<\/span> = xy1[1] - m * xy1[0]\n    <span style=\"color: #00cdcd; font-weight: bold;\">return<\/span> np.array(( m, b ))\n<span style=\"color: #cdcd00;\">x01<\/span>     = np.arange(2)\n<span style=\"color: #cdcd00;\">x01_one<\/span> = nps.glue( nps.transpose(x01), np.ones((2,1)), axis=-1)\n<span style=\"color: #cdcd00;\">y_lowerbound<\/span> = nps.inner(x01_one,\n                         linear_slope_yintercept( np.array((0, Nshifts_leq+1)),\n                                                  np.array((1, 0)) ))\n<span style=\"color: #cdcd00;\">y_upperbound<\/span> = nps.inner(x01_one,\n                         linear_slope_yintercept( np.array((0, Nshifts_max)),\n                                                  np.array((1, Nshifts_leq)) ))\n<span style=\"color: #cdcd00;\">y_lowerbound_check<\/span> = (1-x01) * (Nshifts_leq+1)\n<span style=\"color: #cdcd00;\">y_upperbound_check<\/span> = Nshifts_max - x01*(Nshifts_max-Nshifts_leq)\n\ngp.plot( <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">True value: var_Nshifts_leq4==0, Nshifts&gt;4<\/span>\n         ( np.zeros(i0.shape),\n           Nshifts[i0],\n           <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>(_with     = <span style=\"color: #00cd00;\">'points pt 7 ps 1 lc \"red\"'<\/span>) ),\n         <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">True value: var_Nshifts_leq4==1, Nshifts&lt;=4<\/span>\n         ( np.ones(i1.shape),\n           Nshifts[i1],\n           <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>(_with     = <span style=\"color: #00cd00;\">'points pt 7 ps 1 lc \"red\"'<\/span>) ),\n         <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">False value: var_Nshifts_leq4==1, Nshifts&gt;4<\/span>\n         ( np.ones(i0.shape),\n           Nshifts[i0],\n           <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>(_with     = <span style=\"color: #00cd00;\">'points pt 7 ps 1 lc \"black\"'<\/span>) ),\n         <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">False value: var_Nshifts_leq4==0, Nshifts&lt;=4<\/span>\n         ( np.zeros(i1.shape),\n           Nshifts[i1],\n           <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>(_with     = <span style=\"color: #00cd00;\">'points pt 7 ps 1 lc \"black\"'<\/span>) ),\n\n         ( x01, y_lowerbound, y_upperbound,\n           <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>( _with     = <span style=\"color: #00cd00;\">'filledcurves lc \"green\"'<\/span>,\n                 tuplesize = 3) ),\n         ( x01, nps.cat(y_lowerbound_check, y_upperbound_check),\n           <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>( _with     = <span style=\"color: #00cd00;\">'lines lc \"green\" lw 2'<\/span>,\n                 tuplesize = 2) ),\n\n        unset=(<span style=\"color: #00cd00;\">'grid'<\/span>),\n        _set = (f<span style=\"color: #00cd00;\">'xtics (\"(Nshifts&lt;={Nshifts_leq}) == 0\" 0, \"(Nshifts&lt;={Nshifts_leq}) == 1\" 1)'<\/span>,\n                <span style=\"color: #00cd00;\">'style fill transparent pattern 1'<\/span>),\n        _xrange = (-0.1, 1.1),\n        ylabel = <span style=\"color: #00cd00;\">\"Nshifts\"<\/span>,\n        title = <span style=\"color: #00cd00;\">\"Nshifts inequality variable: linearly separable\"<\/span>,\n        hardcopy = <span style=\"color: #00cd00;\">\"\/tmp\/scheduling-Nshifts-leq.svg\"<\/span>)\n<\/pre>\n<\/div>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2025\/03\/05_shop-scheduling-with-pulp\/scheduling-Nshifts-leq.svg\" alt=\"scheduling-Nshifts-leq.svg\" width=\"80%\" \/>\n<\/p>\n<\/div>\n\n<p>\nSo we can use two linear constraints to make each of these variables work\nproperly. To use these in the benefit function we can use the equality\nconstraint expression from above, or we can use these directly:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-python\"><span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">I want to favor people getting more extra shifts at the start to balance<\/span>\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">things out: somebody getting one more shift on their pile shouldn't take<\/span>\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">shifts away from under-utilized people<\/span>\n<span style=\"color: #cdcd00;\">benefit_boost_leq_bound<\/span> = \\\n    {2: .2,\n     3: .3,\n     4: .4,\n     5: .5}\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">Constrain vars_Nshifts_leq variables to do the right thing<\/span>\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans:\n    <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> b <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> benefit_boost_leq_bound.keys():\n        <span style=\"color: #cdcd00;\">prob<\/span> += (pulp.lpSum([<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s] <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys()])\n                 &gt;= (1 - vars_Nshifts_leq[h][b])*(b+1),\n                 f<span style=\"color: #00cd00;\">\"{h} at least {b} shifts: lower bound\"<\/span>)\n        <span style=\"color: #cdcd00;\">prob<\/span> += (pulp.lpSum([<span style=\"color: #0000ee; font-weight: bold;\">vars<\/span>[h][s] <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> s <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> shifts.keys()])\n                 &lt;= Nshifts_max - vars_Nshifts_leq[h][b]*(Nshifts_max-b),\n                 f<span style=\"color: #00cd00;\">\"{h} at least {b} shifts: upper bound\"<\/span>)\n\n<span style=\"color: #cdcd00;\">benefits<\/span> = <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>()\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> h <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> humans:\n    <span style=\"color: #cdcd00;\">benefits<\/span>[h] = \\\n        ... + \\\n        pulp.lpSum([vars_Nshifts_leq[h][b] * benefit_boost_leq_bound[b] \\\n                    <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> b <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> benefit_boost_leq_bound.keys()])\n<\/pre>\n<\/div>\n\n<p>\nIn <i>this<\/i> scenario, David would get a boost of 0.4 from giving up his 5th shift,\nwhile Judy would lose a boost of 0.2 from getting her 3rd, for a net gain of 0.2\nbenefit points. The exact numbers will need to be adjusted on a case by case\nbasis, but this works.\n<\/p>\n\n<p>\nThe full program, with this and other extra features is available <a href=\"..\/..\/..\/notes\/2025\/03\/05_shop-scheduling-with-pulp\/schedule.py\">here<\/a>.\n<\/p>\n"},{"title":"When are the days getting longer the fastest?","author":{"name":"Dima Kogan"},"link":{"@attributes":{"href":"http:\/\/notes.secretsauce.net\/notes\/2025\/02\/18_when-are-the-days-getting-longer-the-fastest.html"}},"updated":"2025-02-18T18:47:00Z","published":"2025-02-18T18:47:00Z","id":"notes\/2025\/02\/18_when-are-the-days-getting-longer-the-fastest.html","category":[{"@attributes":{"scheme":"\/tags\/data.html","term":"data","label":"data"}},{"@attributes":{"scheme":"\/tags\/vnlog.html","term":"vnlog","label":"vnlog"}},{"@attributes":{"scheme":"\/tags\/tools.html","term":"tools","label":"tools"}}],"content":"<p>\nWe're way past the winter solstice, and approaching the equinox. The\nsun is noticeably staying up later and later every day, which raises\nan obvious question: when are the days getting longer the fastest?\nIntuitively I want to say it should happen at the equinox. But does it\nhappen <i>exactly<\/i> at the equinox? I could read up on all the gory\ndetails of this, or I could just make some plots. I wrote this:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-python\"><span style=\"color: #cdcd00;\">#<\/span><span style=\"color: #cdcd00;\">!\/usr\/bin\/python3<\/span>\n\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> sys\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> datetime\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> astral.sun\n\n<span style=\"color: #cdcd00;\">lat<\/span>  = 34.\n<span style=\"color: #cdcd00;\">year<\/span> = 2025\n\n<span style=\"color: #cdcd00;\">city<\/span> = astral.LocationInfo(latitude=lat, longitude=0)\n\n<span style=\"color: #cdcd00;\">date0<\/span> = datetime.datetime(year, 1, 1)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"# date sunrise sunset length_min\"<\/span>)\n\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> i <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> <span style=\"color: #0000ee; font-weight: bold;\">range<\/span>(365):\n    <span style=\"color: #cdcd00;\">date<\/span> = date0 + datetime.timedelta(days=i)\n\n    <span style=\"color: #cdcd00;\">s<\/span> = astral.sun.sun(city.observer, date=date)\n\n    <span style=\"color: #cdcd00;\">date_sunrise<\/span> = s[<span style=\"color: #00cd00;\">'sunrise'<\/span>]\n    <span style=\"color: #cdcd00;\">date_sunset<\/span>  = s[<span style=\"color: #00cd00;\">'sunset'<\/span>]\n\n    <span style=\"color: #cdcd00;\">date_string<\/span>    = date.strftime(<span style=\"color: #00cd00;\">'%Y-%m-%d'<\/span>)\n    <span style=\"color: #cdcd00;\">sunrise_string<\/span> = date_sunrise.strftime(<span style=\"color: #00cd00;\">'%H:%M'<\/span>)\n    <span style=\"color: #cdcd00;\">sunset_string<\/span>  = date_sunset.strftime (<span style=\"color: #00cd00;\">'%H:%M'<\/span>)\n\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">\"{date_string} {sunrise_string} {sunset_string} {(date_sunset-date_sunrise).total_seconds()\/60}\"<\/span>)\n<\/pre>\n<\/div>\n\n<p>\nThis computes the sunrise and sunset time every day of 2025 at a latitude of\n34degrees (i.e. Los Angeles), and writes out a <a href=\"..\/..\/..\/notes\/2025\/02\/18_when-are-the-days-getting-longer-the-fastest\/sunrise-sunset.vnl\">log file<\/a> (using the <a href=\"https:\/\/github.com\/dkogan\/vnlog\">vnlog<\/a>\nformat).\n<\/p>\n\n<p>\nLet's plot it:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-sh\">&lt; sunrise-sunset.vnl                   <span style=\"color: #00cd00;\">\\<\/span>\n  vnl-filter -p date,<span style=\"color: #cdcd00;\">l<\/span>=<span style=\"color: #00cd00;\">'length_min\/60'<\/span> <span style=\"color: #00cd00;\">\\<\/span>\n| feedgnuplot                          <span style=\"color: #00cd00;\">\\<\/span>\n  --set <span style=\"color: #00cd00;\">'format x \"%b %d\"'<\/span>             <span style=\"color: #00cd00;\">\\<\/span>\n  --domain                             <span style=\"color: #00cd00;\">\\<\/span>\n  --timefmt <span style=\"color: #00cd00;\">'%Y-%m-%d'<\/span>                 <span style=\"color: #00cd00;\">\\<\/span>\n  --lines                              <span style=\"color: #00cd00;\">\\<\/span>\n  --ylabel <span style=\"color: #00cd00;\">'Day length (hours)'<\/span>        <span style=\"color: #00cd00;\">\\<\/span>\n  --hardcopy day-length.svg\n<\/pre>\n<\/div>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2025\/02\/18_when-are-the-days-getting-longer-the-fastest\/day-length.svg\" alt=\"day-length.svg\" width=\"70%\" \/>\n<\/p>\n<\/div>\n\n<p>\nWell that makes sense. When are the days the longest\/shortest?\n<\/p>\n\n<pre class=\"example\">\n$ &lt; sunrise-sunset.vnl vnl-sort -grk length_min | head -n2 | vnl-align\n\n#  date    sunrise sunset     length_min   \n2025-06-21 04:49   19:14  864.8543702000001\n\n\n$ &lt; sunrise-sunset.vnl vnl-sort -gk length_min | head -n2 | vnl-align\n\n#  date    sunrise sunset     length_min   \n2025-12-21 07:01   16:54  592.8354265166668\n<\/pre>\n\n<p>\nThose are the solstices, as expected. Now let's look at the time gained\/lost\neach day:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-sh\">$ &lt; sunrise-sunset.vnl                                  <span style=\"color: #00cd00;\">\\<\/span>\n  vnl-filter -p date,<span style=\"color: #cdcd00;\">d<\/span>=<span style=\"color: #00cd00;\">'diff(length_min)'<\/span>               <span style=\"color: #00cd00;\">\\<\/span>\n| vnl-filter --has d                                    <span style=\"color: #00cd00;\">\\<\/span>\n| feedgnuplot                                           <span style=\"color: #00cd00;\">\\<\/span>\n  --set <span style=\"color: #00cd00;\">'format x \"%b %d\"'<\/span>                              <span style=\"color: #00cd00;\">\\<\/span>\n  --domain                                              <span style=\"color: #00cd00;\">\\<\/span>\n  --timefmt <span style=\"color: #00cd00;\">'%Y-%m-%d'<\/span>                                  <span style=\"color: #00cd00;\">\\<\/span>\n  --lines                                               <span style=\"color: #00cd00;\">\\<\/span>\n  --ylabel <span style=\"color: #00cd00;\">'Daytime gained from the previous day (min)'<\/span> <span style=\"color: #00cd00;\">\\<\/span>\n  --hardcopy gain.svg\n<\/pre>\n<\/div>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2025\/02\/18_when-are-the-days-getting-longer-the-fastest\/gain.svg\" alt=\"gain.svg\" width=\"70%\" \/>\n<\/p>\n<\/div>\n\n<p>\nLooks vaguely sinusoidal, like the last plot. And looks like we gain\/lost as\nmost ~2 minutes each day. When does the gain peak?\n<\/p>\n\n<pre class=\"example\">\n$ &lt; sunrise-sunset.vnl vnl-filter -p date,d='diff(length_min)' | vnl-filter --has d | vnl-sort -grk d | head -n2 | vnl-align\n\n#  date       d   \n2025-03-19 2.13167\n\n\n$ &lt; sunrise-sunset.vnl vnl-filter -p date,d='diff(length_min)' | vnl-filter --has d | vnl-sort -gk d | head -n2 | vnl-align\n\n#  date        d   \n2025-09-25 -2.09886\n<\/pre>\n\n<p>\n<i>Not<\/i> at the equinoxes! The fastest gain is a few days before the equinox and\nthe fastest loss a few days after.\n<\/p>\n"},{"title":"Strava track filtering validation","author":{"name":"Dima Kogan"},"link":{"@attributes":{"href":"http:\/\/notes.secretsauce.net\/notes\/2024\/11\/30_strava-track-filtering-validation.html"}},"updated":"2024-11-30T14:48:00Z","published":"2024-11-30T14:48:00Z","id":"notes\/2024\/11\/30_strava-track-filtering-validation.html","category":{"@attributes":{"scheme":"\/tags\/data.html","term":"data","label":"data"}},"content":"<p>\nAfter years of seeing people's strava tracks, I became convinced that strava\ndoesn't sufficiently filter the data, resulting in over-estimated effort\nnumbers. Today I did a bit of lazy analysis, and half-confirmed this: in the one\ncase I looked at, strava reported reasonable elevation gain numbers, but greatly\noverestimated the distance traveled.\n<\/p>\n\n<p>\nI looked at a single gps track of a long bike ride. This was uploaded to strava\nmanually, as a <code>.gpx<\/code> file. I can imagine that something different happens if\nyou use the strava app or some device that integrates with the service: the\nfiltering might happen before the data hits the server, and the server could\nthen decide to not apply any additional filtering.\n<\/p>\n\n<p>\nI processed the data with a simple hysteretic filter, ignoring small changes in\nposition and elevation, trying out different thresholds in the process. I\ncompletely ignore the timestamps, and only look at the differences between\nsuccessive points. This handles the usual GPS noise; it does <i>not<\/i> handle GPS\njumps, which I completely ignore in this analysis. Ignoring these would produce\ninflated elevation\/gain numbers, but I'm working with a looong track, so\nhopefully this is a small effect.\n<\/p>\n\n<p>\nClearly this is not scientific, but it's something.\n<\/p>\n\n<div id=\"outline-container-sec-1\" class=\"outline-2\">\n<h2 id=\"sec-1\">The code<\/h2>\n<div class=\"outline-text-2\" id=\"text-1\">\n<p>\nParsing <code>.gpx<\/code> is slow (this is a <i>big<\/i> file), so I cache that into a <a href=\"https:\/\/github.com\/dkogan\/vnlog\"><code>.vnl<\/code><\/a>:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-python\"><span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> sys\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> gpxpy\n\n<span style=\"color: #cdcd00;\">filename_in<\/span>  = <span style=\"color: #00cd00;\">'INPUT.gpx'<\/span>\n<span style=\"color: #cdcd00;\">filename_out<\/span> = <span style=\"color: #00cd00;\">'OUTPUT.gpx'<\/span>\n\n<span style=\"color: #00cdcd; font-weight: bold;\">with<\/span> <span style=\"color: #0000ee; font-weight: bold;\">open<\/span>(filename_in, <span style=\"color: #00cd00;\">'r'<\/span>) <span style=\"color: #00cdcd; font-weight: bold;\">as<\/span> f:\n    <span style=\"color: #cdcd00;\">gpx<\/span> = gpxpy.parse(f)\n\n<span style=\"color: #cdcd00;\">f_out<\/span> = <span style=\"color: #0000ee; font-weight: bold;\">open<\/span>(filename_out, <span style=\"color: #00cd00;\">'w'<\/span>)\n\n<span style=\"color: #cdcd00;\">tracks<\/span> = gpx.tracks\n<span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> <span style=\"color: #0000ee; font-weight: bold;\">len<\/span>(tracks) != 1:\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"I want just one track\"<\/span>, <span style=\"color: #0000ee; font-weight: bold;\">file<\/span>=sys.stderr)\n    sys.<span style=\"color: #cd00cd;\">exit<\/span>(1)\n<span style=\"color: #cdcd00;\">track<\/span> = tracks[0]\n\n<span style=\"color: #cdcd00;\">segments<\/span> = track.segments\n<span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> <span style=\"color: #0000ee; font-weight: bold;\">len<\/span>(segments) != 1:\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"I want just one segment\"<\/span>, <span style=\"color: #0000ee; font-weight: bold;\">file<\/span>=sys.stderr)\n    sys.<span style=\"color: #cd00cd;\">exit<\/span>(1)\n<span style=\"color: #cdcd00;\">segment<\/span> = segments[0]\n\n<span style=\"color: #cdcd00;\">time0<\/span> = segment.points[0].time\n<span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(<span style=\"color: #00cd00;\">\"# time lat lon ele_m\"<\/span>)\n<span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> p <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> segment.points:\n    <span style=\"color: #00cdcd; font-weight: bold;\">print<\/span>(f<span style=\"color: #00cd00;\">\"{(p.time - time0).seconds} {p.latitude} {p.longitude} {p.elevation}\"<\/span>,\n          <span style=\"color: #0000ee; font-weight: bold;\">file<\/span> = f_out)\n<\/pre>\n<\/div>\n\n<p>\nAnd I then process this data with the different filters (this is a silly Python\nloop, and is slow):\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-python\"><span style=\"color: #cdcd00;\">#<\/span><span style=\"color: #cdcd00;\">!\/usr\/bin\/python3<\/span>\n\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> sys\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> numpy <span style=\"color: #00cdcd; font-weight: bold;\">as<\/span> np\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> numpysane <span style=\"color: #00cdcd; font-weight: bold;\">as<\/span> nps\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> gnuplotlib <span style=\"color: #00cdcd; font-weight: bold;\">as<\/span> gp\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> vnlog\n<span style=\"color: #00cdcd; font-weight: bold;\">import<\/span> pyproj\n\n<span style=\"color: #cdcd00;\">geod<\/span> = <span style=\"color: #cd00cd;\">None<\/span>\n<span style=\"color: #00cdcd; font-weight: bold;\">def<\/span> <span style=\"color: #0000ee; font-weight: bold;\">dist_ft<\/span>(lat0,lon0, lat1,lon1):\n\n    <span style=\"color: #00cdcd; font-weight: bold;\">global<\/span> geod\n    <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> geod <span style=\"color: #00cdcd; font-weight: bold;\">is<\/span> <span style=\"color: #cd00cd;\">None<\/span>:\n        <span style=\"color: #cdcd00;\">geod<\/span> = pyproj.Geod(ellps=<span style=\"color: #00cd00;\">'WGS84'<\/span>)\n    <span style=\"color: #00cdcd; font-weight: bold;\">return<\/span> \\\n        geod.inv(lon0,lat0, lon1,lat1)[2] * 100.\/2.54\/12.\n\n\n\n\n<span style=\"color: #cdcd00;\">f<\/span> = <span style=\"color: #00cd00;\">'OUTPUT.gpx'<\/span>\n\n<span style=\"color: #cdcd00;\">track<\/span>,<span style=\"color: #cdcd00;\">list_keys<\/span>,<span style=\"color: #cdcd00;\">dict_key_index<\/span> = \\\n    vnlog.slurp(f)\n\n<span style=\"color: #cdcd00;\">t<\/span>      = track[:,dict_key_index[<span style=\"color: #00cd00;\">'time'<\/span> ]]\n<span style=\"color: #cdcd00;\">lat<\/span>    = track[:,dict_key_index[<span style=\"color: #00cd00;\">'lat'<\/span>  ]]\n<span style=\"color: #cdcd00;\">lon<\/span>    = track[:,dict_key_index[<span style=\"color: #00cd00;\">'lon'<\/span>  ]]\n<span style=\"color: #cdcd00;\">ele_ft<\/span> = track[:,dict_key_index[<span style=\"color: #00cd00;\">'ele_m'<\/span>]] * 100.\/2.54\/12.\n\n\n\n<span style=\"color: #00cd00;\">@nps.broadcast_define<\/span>( ( (), ()),\n                       (2,))\n<span style=\"color: #00cdcd; font-weight: bold;\">def<\/span> <span style=\"color: #0000ee; font-weight: bold;\">filter_track<\/span>(ele_hysteresis_ft,\n                 dxy_hysteresis_ft):\n\n    <span style=\"color: #cdcd00;\">dist<\/span>        = 0.0\n    <span style=\"color: #cdcd00;\">ele_gain_ft<\/span> = 0.0\n\n    <span style=\"color: #cdcd00;\">lon_accepted<\/span> = <span style=\"color: #cd00cd;\">None<\/span>\n    <span style=\"color: #cdcd00;\">lat_accepted<\/span> = <span style=\"color: #cd00cd;\">None<\/span>\n    <span style=\"color: #cdcd00;\">ele_accepted<\/span> = <span style=\"color: #cd00cd;\">None<\/span>\n\n    <span style=\"color: #00cdcd; font-weight: bold;\">for<\/span> i <span style=\"color: #00cdcd; font-weight: bold;\">in<\/span> <span style=\"color: #0000ee; font-weight: bold;\">range<\/span>(<span style=\"color: #0000ee; font-weight: bold;\">len<\/span>(lat)):\n\n        <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> ele_accepted <span style=\"color: #00cdcd; font-weight: bold;\">is<\/span> <span style=\"color: #00cdcd; font-weight: bold;\">not<\/span> <span style=\"color: #cd00cd;\">None<\/span>:\n            <span style=\"color: #cdcd00;\">dxy_here<\/span>  = dist_ft(lat_accepted,lon_accepted, lat[i],lon[i])\n            <span style=\"color: #cdcd00;\">dele_here<\/span> = np.<span style=\"color: #0000ee; font-weight: bold;\">abs<\/span>( ele_ft[i] - ele_accepted )\n\n            <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> dxy_here &lt; dxy_hysteresis_ft <span style=\"color: #00cdcd; font-weight: bold;\">and<\/span> dele_here &lt; ele_hysteresis_ft:\n                <span style=\"color: #00cdcd; font-weight: bold;\">continue<\/span>\n\n            <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> ele_ft[i] &gt; ele_accepted:\n                <span style=\"color: #cdcd00;\">ele_gain_ft<\/span> += dele_here;\n\n            <span style=\"color: #cdcd00;\">dist<\/span> += np.sqrt(dele_here * dele_here +\n                            dxy_here  * dxy_here)\n\n        <span style=\"color: #cdcd00;\">lon_accepted<\/span> = lon[i]\n        <span style=\"color: #cdcd00;\">lat_accepted<\/span> = lat[i]\n        <span style=\"color: #cdcd00;\">ele_accepted<\/span> = ele_ft[i]\n\n    <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">lose the last point. It simply doesn't matter<\/span>\n\n    <span style=\"color: #cdcd00;\">dist_mi<\/span> = dist \/ 5280.\n    <span style=\"color: #00cdcd; font-weight: bold;\">return<\/span> np.array((ele_gain_ft, dist_mi))\n\n\n\n\n<span style=\"color: #cdcd00;\">Nele_hysteresis_ft<\/span>    = 20\n<span style=\"color: #cdcd00;\">ele_hysteresis0_ft<\/span>    = 5\n<span style=\"color: #cdcd00;\">ele_hysteresis1_ft<\/span>    = 100\n<span style=\"color: #cdcd00;\">ele_hysteresis_ft_all<\/span> = np.linspace(ele_hysteresis0_ft,\n                                    ele_hysteresis1_ft,\n                                    Nele_hysteresis_ft)\n\n<span style=\"color: #cdcd00;\">Ndxy_hysteresis_ft<\/span> = 20\n<span style=\"color: #cdcd00;\">dxy_hysteresis0_ft<\/span> = 5\n<span style=\"color: #cdcd00;\">dxy_hysteresis1_ft<\/span> = 1000\n<span style=\"color: #cdcd00;\">dxy_hysteresis_ft<\/span>  = np.linspace(dxy_hysteresis0_ft,\n                                 dxy_hysteresis1_ft,\n                                 Ndxy_hysteresis_ft)\n\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">shape (Nele,Ndxy,2)<\/span>\n<span style=\"color: #cdcd00;\">gain<\/span>,<span style=\"color: #cdcd00;\">distance<\/span> = \\\n    nps.mv( filter_track( nps.dummy(ele_hysteresis_ft_all,-1),\n                          dxy_hysteresis_ft),\n            -1,0 )\n\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">Stolen from mrcal<\/span>\n<span style=\"color: #00cdcd; font-weight: bold;\">def<\/span> <span style=\"color: #0000ee; font-weight: bold;\">options_heatmap_with_contours<\/span>( plotoptions, <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">we update this on output<\/span>\n\n                                   *,\n                                   contour_min           = 0,\n                                   contour_max,\n                                   contour_increment     = <span style=\"color: #cd00cd;\">None<\/span>,\n                                   do_contours           = <span style=\"color: #cd00cd;\">True<\/span>,\n                                   contour_labels_styles = <span style=\"color: #00cd00;\">'boxed'<\/span>,\n                                   contour_labels_font   = <span style=\"color: #cd00cd;\">None<\/span>):\n    r<span style=\"color: #00cd00;\">'''Update plotoptions, return curveoptions for a contoured heat map'''<\/span>\n\n    gp.add_plot_option(plotoptions,\n                       <span style=\"color: #00cd00;\">'set'<\/span>,\n                       (<span style=\"color: #00cd00;\">'view equal xy'<\/span>,\n                        <span style=\"color: #00cd00;\">'view map'<\/span>))\n\n    <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> do_contours:\n        <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> contour_increment <span style=\"color: #00cdcd; font-weight: bold;\">is<\/span> <span style=\"color: #cd00cd;\">None<\/span>:\n            <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">Compute a \"nice\" contour increment. I pick a round number that gives<\/span>\n            <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">me a reasonable number of contours<\/span>\n\n            <span style=\"color: #cdcd00;\">Nwant<\/span> = 10\n            <span style=\"color: #cdcd00;\">increment<\/span> = (contour_max - contour_min)\/Nwant\n\n            <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">I find the nearest 1eX or 2eX or 5eX<\/span>\n            <span style=\"color: #cdcd00;\">base10_floor<\/span> = np.power(10., np.floor(np.log10(increment)))\n\n            <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">Look through the options, and pick the best one<\/span>\n            <span style=\"color: #cdcd00;\">m<\/span>   = np.array((1., 2., 5., 10.))\n            <span style=\"color: #cdcd00;\">err<\/span> = np.<span style=\"color: #0000ee; font-weight: bold;\">abs<\/span>(m * base10_floor - increment)\n            <span style=\"color: #cdcd00;\">contour_increment<\/span> = -m[ np.argmin(err) ] * base10_floor\n\n        gp.add_plot_option(plotoptions,\n                           <span style=\"color: #00cd00;\">'set'<\/span>,\n                           (<span style=\"color: #00cd00;\">'key box opaque'<\/span>,\n                            <span style=\"color: #00cd00;\">'style textbox opaque'<\/span>,\n                            <span style=\"color: #00cd00;\">'contour base'<\/span>,\n                            f<span style=\"color: #00cd00;\">'cntrparam levels incremental {contour_max},{contour_increment},{contour_min}'<\/span>))\n\n        <span style=\"color: #00cdcd; font-weight: bold;\">if<\/span> contour_labels_font <span style=\"color: #00cdcd; font-weight: bold;\">is<\/span> <span style=\"color: #00cdcd; font-weight: bold;\">not<\/span> <span style=\"color: #cd00cd;\">None<\/span>:\n            gp.add_plot_option(plotoptions,\n                               <span style=\"color: #00cd00;\">'set'<\/span>,\n                               f<span style=\"color: #00cd00;\">'cntrlabel format \"%d\" font \"{contour_labels_font}\"'<\/span> )\n        <span style=\"color: #00cdcd; font-weight: bold;\">else<\/span>:\n            gp.add_plot_option(plotoptions,\n                               <span style=\"color: #00cd00;\">'set'<\/span>,\n                               f<span style=\"color: #00cd00;\">'cntrlabel format \"%.0f\"'<\/span> )\n\n        <span style=\"color: #cdcd00;\">plotoptions<\/span>[<span style=\"color: #00cd00;\">'cbrange'<\/span>] = [contour_min, contour_max]\n\n        <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">I plot 3 times:<\/span>\n        <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">- to make the heat map<\/span>\n        <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">- to make the contours<\/span>\n        <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">- to make the contour labels<\/span>\n        <span style=\"color: #cdcd00;\">_with<\/span> = np.array((<span style=\"color: #00cd00;\">'image'<\/span>,\n                          <span style=\"color: #00cd00;\">'lines nosurface'<\/span>,\n                          f<span style=\"color: #00cd00;\">'labels {contour_labels_styles} nosurface'<\/span>))\n    <span style=\"color: #00cdcd; font-weight: bold;\">else<\/span>:\n        gp.add_plot_option(plotoptions, <span style=\"color: #00cd00;\">'unset'<\/span>, <span style=\"color: #00cd00;\">'key'<\/span>)\n        <span style=\"color: #cdcd00;\">_with<\/span> = <span style=\"color: #00cd00;\">'image'<\/span>\n\n    <span style=\"color: #cdcd00;\">using<\/span> = \\\n        f<span style=\"color: #00cd00;\">'({dxy_hysteresis0_ft}+$1*{float(dxy_hysteresis1_ft-dxy_hysteresis0_ft)\/(Ndxy_hysteresis_ft-1)}):'<\/span> + \\\n        f<span style=\"color: #00cd00;\">'({ele_hysteresis0_ft}+$2*{float(ele_hysteresis1_ft-ele_hysteresis0_ft)\/(Nele_hysteresis_ft-1)}):3'<\/span>\n    <span style=\"color: #cdcd00;\">plotoptions<\/span>[<span style=\"color: #00cd00;\">'_3d'<\/span>]     = <span style=\"color: #cd00cd;\">True<\/span>\n    <span style=\"color: #cdcd00;\">plotoptions<\/span>[<span style=\"color: #00cd00;\">'_xrange'<\/span>] = [dxy_hysteresis0_ft,dxy_hysteresis1_ft]\n    <span style=\"color: #cdcd00;\">plotoptions<\/span>[<span style=\"color: #00cd00;\">'_yrange'<\/span>] = [ele_hysteresis0_ft,ele_hysteresis1_ft]\n    <span style=\"color: #cdcd00;\">plotoptions<\/span>[<span style=\"color: #00cd00;\">'ascii'<\/span>]   = <span style=\"color: #cd00cd;\">True<\/span> <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">needed for using to work<\/span>\n\n    gp.add_plot_option(plotoptions, <span style=\"color: #00cd00;\">'unset'<\/span>, <span style=\"color: #00cd00;\">'grid'<\/span>)\n\n    <span style=\"color: #00cdcd; font-weight: bold;\">return<\/span> \\\n        <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>( tuplesize=3,\n              legend = <span style=\"color: #00cd00;\">\"\"<\/span>, <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">needed to force contour labels<\/span>\n              using = using,\n              _with=_with)\n\n\n\n\n<span style=\"color: #cdcd00;\">contour_granularity<\/span> = 1000\n<span style=\"color: #cdcd00;\">plotoptions<\/span> = <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>()\n<span style=\"color: #cdcd00;\">curveoptions<\/span> = \\\n    options_heatmap_with_contours( plotoptions, <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">we update this on output<\/span>\n                                   <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">round down to the nearest contour_granularity<\/span>\n                                   contour_min = (np.<span style=\"color: #0000ee; font-weight: bold;\">min<\/span>(gain) \/\/ contour_granularity)*contour_granularity,\n                                   <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">round up to the nearest contour_granularity<\/span>\n                                   contour_max = ((np.<span style=\"color: #0000ee; font-weight: bold;\">max<\/span>(gain) + (contour_granularity-1)) \/\/ contour_granularity) * contour_granularity,\n                                   do_contours = <span style=\"color: #cd00cd;\">True<\/span>)\ngp.add_plot_option(plotoptions, <span style=\"color: #00cd00;\">'unset'<\/span>, <span style=\"color: #00cd00;\">'key'<\/span>)\ngp.add_plot_option(plotoptions, <span style=\"color: #00cd00;\">'set'<\/span>, <span style=\"color: #00cd00;\">'size square'<\/span>)\ngp.plot(gain,\n        xlabel  = <span style=\"color: #00cd00;\">\"Distance hysteresis (ft)\"<\/span>,\n        ylabel  = <span style=\"color: #00cd00;\">\"Elevation hysteresis (ft)\"<\/span>,\n        cblabel = <span style=\"color: #00cd00;\">\"Elevation gain (ft)\"<\/span>,\n        wait = <span style=\"color: #cd00cd;\">True<\/span>,\n        **curveoptions,\n        **plotoptions,\n        title    = <span style=\"color: #00cd00;\">'Computed gain vs filtering parameters'<\/span>)\n\n\n<span style=\"color: #cdcd00;\">contour_granularity<\/span> = 10\n<span style=\"color: #cdcd00;\">plotoptions<\/span> = <span style=\"color: #0000ee; font-weight: bold;\">dict<\/span>()\n<span style=\"color: #cdcd00;\">curveoptions<\/span> = \\\n    options_heatmap_with_contours( plotoptions, <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">we update this on output<\/span>\n                                   <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">round down to the nearest contour_granularity<\/span>\n                                   contour_min = (np.<span style=\"color: #0000ee; font-weight: bold;\">min<\/span>(distance) \/\/ contour_granularity)*contour_granularity,\n                                   <span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">round up to the nearest contour_granularity<\/span>\n                                   contour_max = ((np.<span style=\"color: #0000ee; font-weight: bold;\">max<\/span>(distance) + (contour_granularity-1)) \/\/ contour_granularity) * contour_granularity,\n                                   do_contours = <span style=\"color: #cd00cd;\">True<\/span>)\ngp.add_plot_option(plotoptions, <span style=\"color: #00cd00;\">'unset'<\/span>, <span style=\"color: #00cd00;\">'key'<\/span>)\ngp.add_plot_option(plotoptions, <span style=\"color: #00cd00;\">'set'<\/span>, <span style=\"color: #00cd00;\">'size square'<\/span>)\ngp.plot(distance,\n        xlabel  = <span style=\"color: #00cd00;\">\"Distance hysteresis (ft)\"<\/span>,\n        ylabel  = <span style=\"color: #00cd00;\">\"Elevation hysteresis (ft)\"<\/span>,\n        cblabel = <span style=\"color: #00cd00;\">\"Distance (miles)\"<\/span>,\n        wait = <span style=\"color: #cd00cd;\">True<\/span>,\n        **curveoptions,\n        **plotoptions,\n        title    = <span style=\"color: #00cd00;\">'Computed distance vs filtering parameters'<\/span>)\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n\n<div id=\"outline-container-sec-2\" class=\"outline-2\">\n<h2 id=\"sec-2\">Results: gain<\/h2>\n<div class=\"outline-text-2\" id=\"text-2\">\n<p>\nStrava says the gain was 46307ft. The analysis says:\n<\/p>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2024\/11\/30_strava-track-filtering-validation\/strava-gain.png\" alt=\"strava-gain.png\" width=\"70%\" \/>\n<\/p>\n<\/div>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2024\/11\/30_strava-track-filtering-validation\/strava-gain-zoom.png\" alt=\"strava-gain-zoom.png\" width=\"70%\" \/>\n<\/p>\n<\/div>\n\n<p>\nThese show the filtered gain for different values of the distance and gain\nhysteresis thresholds. The same data is shown at diffent zoom levels. There's no\nclear sweet spot, but we get 46307ft with a reasonable amount of filtering.\nMaybe 46307ft is a bit low even.\n<\/p>\n<\/div>\n<\/div>\n\n<div id=\"outline-container-sec-3\" class=\"outline-2\">\n<h2 id=\"sec-3\">Results: distance<\/h2>\n<div class=\"outline-text-2\" id=\"text-3\">\n<p>\nStrava says the distance covered was 322 miles. The analysis says:\n<\/p>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2024\/11\/30_strava-track-filtering-validation\/strava-distance.png\" alt=\"strava-distance.png\" width=\"70%\" \/>\n<\/p>\n<\/div>\n\n\n<div class=\"figure\">\n<p><img src=\"..\/..\/..\/notes\/2024\/11\/30_strava-track-filtering-validation\/strava-distance-zoom.png\" alt=\"strava-distance-zoom.png\" width=\"70%\" \/>\n<\/p>\n<\/div>\n\n<p>\nOnce again, there's no sweet spot, but we get 322 miles only if we apply no\nfiltering at all. That's clearly too high, and is not reasonable. From the map\n(and from other people's strava routes) the true distance is closer to 305\nmiles. Why <i>those<\/i> people's strava numbers are more believable is anybody's\nguess.\n<\/p>\n<\/div>\n<\/div>\n"},{"title":"GNU Make: details regarding intermediate files","author":{"name":"Dima Kogan"},"link":{"@attributes":{"href":"http:\/\/notes.secretsauce.net\/notes\/2024\/09\/08_gnu-make-details-regarding-intermediate-files.html"}},"updated":"2024-09-08T12:31:00Z","published":"2024-09-08T12:31:00Z","id":"notes\/2024\/09\/08_gnu-make-details-regarding-intermediate-files.html","category":{"@attributes":{"scheme":"\/tags\/tools.html","term":"tools","label":"tools"}},"content":"<p>\nCheck this out!\n<\/p>\n\n<p>\nSuppose I have this <code>Makefile<\/code>:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-makefile\"><span style=\"color: #0000ee; font-weight: bold;\">a<\/span>: b\n      touch <span style=\"color: #0000ee; font-weight: bold;\">$<\/span><span style=\"color: #cd00cd; font-weight: bold;\">@<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">b<\/span>:\n      touch <span style=\"color: #0000ee; font-weight: bold;\">$<\/span><span style=\"color: #cd00cd; font-weight: bold;\">@<\/span>\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">A common chain of build steps<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">%-GENERATED.c<\/span>: %-generate\n      touch <span style=\"color: #0000ee; font-weight: bold;\">$<\/span><span style=\"color: #cd00cd; font-weight: bold;\">@<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">%.o<\/span>: %.c\n      touch <span style=\"color: #0000ee; font-weight: bold;\">$<\/span><span style=\"color: #cd00cd; font-weight: bold;\">@<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">%.so<\/span>: %-GENERATED.o\n      touch <span style=\"color: #0000ee; font-weight: bold;\">$<\/span><span style=\"color: #cd00cd; font-weight: bold;\">@<\/span>\n<span style=\"color: #0000ee; font-weight: bold;\">xxx-GENERATED.o<\/span>: <span style=\"color: #cdcd00;\">CFLAGS<\/span> += adsf\n\n<span style=\"color: #cdcd00;\"># <\/span><span style=\"color: #cdcd00;\">Imitates .d files created with \"gcc -MMD\". Does not exist on the initial build<\/span>\nifneq ($(<span style=\"color: #cdcd00;\">wildcard<\/span> xxx.so),)\n<span style=\"color: #0000ee; font-weight: bold;\">xxx-GENERATED.o<\/span>: xxx-GENERATED.c\nendif\n<\/pre>\n<\/div>\n\n<p>\nThis is all very simple build-system stuff. Let's see how it works:\n<\/p>\n\n<pre class=\"example\">\n$ rm -rf a b xxx-GENERATED.c xxx-GENERATED.o xxx.so\n  [start from a clean slate]\n\n$ touch xxx-generate xxx.h\n  [Files that would be available in a project exist; xxx-generate is some tool]\n  [that would generate xxx-GENERATED.c                                        ]\n\n$ touch a\n  [\"a\" exists but the file \"b\" it depends on does not]\n\n$ make a xxx.so\n\n  touch b\n  touch a\n  touch xxx-GENERATED.c\n  touch xxx-GENERATED.o\n  touch xxx.so\n  rm xxx-GENERATED.c\n\n  [It built everything, but then deleted xxx-GENERATED.c]\n\n$ make a xxx.so\n\n  remake: 'a' is up to date.\n  touch xxx-GENERATED.c\n  touch xxx-GENERATED.o\n  touch xxx.so\n\n  [It knew to not rebuild \"a\", but the missing xxx-GENERATED.c caused it to]\n  [re-build stuff                                                          ]\n<\/pre>\n\n<p>\nWell that's not good. What if we add <code>.SECONDARY:<\/code> to the end of the <code>Makefile<\/code>\nto mark everything as a secondary file?\n<\/p>\n\n<pre class=\"example\">\n$ rm -rf a b xxx-GENERATED.c xxx-GENERATED.o xxx.so\n$ touch xxx-generate xxx.h\n$ touch a\n\n$ make a xxx.so\n\n  remake: 'a' is up to date.\n  touch xxx-GENERATED.c\n  touch xxx-GENERATED.o\n  touch xxx.so\n\n  [It didn't bother rebuilding \"a\" even though its prerequisites \"b\" doesn't]\n  [exist. But it didn't delete the xxx-GENERATED.c at least                 ]\n\n$ make a xxx.so\n\n  remake: 'a' is up to date.\n  remake: 'xxx.so' is up to date.\n\n  [It knew to not rebuild anything. Great.]\n<\/pre>\n\n<p>\nSo it doesn't work right with or without <code>.SECONDARY:<\/code>, but it's much closer\nwith it. The solution is to mark everything as <i>not<\/i> an intermediate file.\nmrbuild cannot do this without a bleeding-edge version of <code>GNU Make<\/code>, but users\nof mrbuild <i>can<\/i> do this by explicitly mentioning specific files in rules. This\nwould suffice:\n<\/p>\n\n<div class=\"org-src-container\">\n\n<pre class=\"src src-makefile\"><span style=\"color: #0000ee; font-weight: bold;\">___dummy___<\/span>: file1 file2\n<\/pre>\n<\/div>\n\n<p>\nDetailed notes are in a <a href=\"https:\/\/github.com\/dkogan\/mrbuild\/commit\/87a2be281d7d7a27d501196ca4ec5e82324f4095\">commit in mrbuild<\/a> (mrbuild 1.13) and in a <a href=\"https:\/\/lore.kernel.org\/lkml\/Y6WUlth8KrR6EcsI@bergen.fjasle.eu\/T\/\">post to LKML\nby Masahiro Yamada<\/a>.\n<\/p>\n"}]}