{"title":"St\u00e9phane on Locomotion - Comments: Quadratic programming in Python","link":[{"@attributes":{"href":"https:\/\/scaron.info\/blog\/quadratic-programming-in-python.html\/","rel":"alternate"}},{"@attributes":{"href":"https:\/\/scaron.info\/feeds\/comment.quadratic-programming-in-python.atom.xml","rel":"self"}}],"id":"https:\/\/scaron.info\/blog\/quadratic-programming-in-python.html\/","updated":"2023-01-04T10:36:00+01:00","entry":[{"title":"Posted by: St\u00e9phane","link":{"@attributes":{"href":"https:\/\/scaron.info\/blog\/quadratic-programming-in-python.html\/#comment-reply-cvxopt-large-unconstrained-dense","rel":"alternate"}},"published":"2023-01-04T10:36:00+01:00","updated":"2023-01-04T10:36:00+01:00","author":{"name":"St\u00e9phane"},"id":"tag:scaron.info,2023-01-04:\/blog\/quadratic-programming-in-python.html\/\/comment-reply-cvxopt-large-unconstrained-dense","summary":"<p>For CVXOPT not really. According to the feedback in that issue, the solver is unlikely to move to a different version of BLAS, which makes some sense from a maintainer perspective as the project is in a stable state.<\/p>\n<p>In that case, I would probably switch gears to ProxQP, which \u2026<\/p>","content":"<p>For CVXOPT not really. According to the feedback in that issue, the solver is unlikely to move to a different version of BLAS, which makes some sense from a maintainer perspective as the project is in a stable state.<\/p>\n<p>In that case, I would probably switch gears to ProxQP, which is under active development (especially its sparse backend) and uses more recent software. (Disclaimer: I've just started working in the same team as the ProxQP developers \ud83d\ude09) If you face a limitation and open an issue there, ideally with a sample problem illustrating what doesn't work as expected, developers are more likely to act on it.<\/p>","category":{"@attributes":{"term":"misc"}}},{"title":"Posted by: Andreas","link":{"@attributes":{"href":"https:\/\/scaron.info\/blog\/quadratic-programming-in-python.html\/#comment-question-cvxopt-large-unconstrained-dense","rel":"alternate"}},"published":"2023-01-03T16:07:00-08:00","updated":"2023-01-03T16:07:00-08:00","author":{"name":"Andreas"},"id":"tag:scaron.info,2023-01-03:\/blog\/quadratic-programming-in-python.html\/\/comment-question-cvxopt-large-unconstrained-dense","summary":"<p>Thanks very much for the explanation. Indeed it seems that cvxopt is fastest for this problem. Unfortunately, I realized however that cvxopt cannot handle (very) large matrices with more than <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><semantics><mrow><msup><mn>2<\/mn><mn>32<\/mn><\/msup><mo>\u2212<\/mo><mn>1<\/mn><\/mrow><annotation encoding=\"application\/x-tex\">\\def\\bfA{\\boldsymbol{A}}\n\\def\\bfB{\\boldsymbol{B}}\n\\def\\bfC{\\boldsymbol{C}}\n\\def\\bfD{\\boldsymbol{D}}\n\\def \u2026<\/annotation><\/semantics><\/math><\/span><\/span><\/p>","content":"<p>Thanks very much for the explanation. Indeed it seems that cvxopt is fastest for this problem. Unfortunately, I realized however that cvxopt cannot handle (very) large matrices with more than <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><semantics><mrow><msup><mn>2<\/mn><mn>32<\/mn><\/msup><mo>\u2212<\/mo><mn>1<\/mn><\/mrow><annotation encoding=\"application\/x-tex\">\\def\\bfA{\\boldsymbol{A}}\n\\def\\bfB{\\boldsymbol{B}}\n\\def\\bfC{\\boldsymbol{C}}\n\\def\\bfD{\\boldsymbol{D}}\n\\def\\bfE{\\boldsymbol{E}}\n\\def\\bfF{\\boldsymbol{F}}\n\\def\\bfG{\\boldsymbol{G}}\n\\def\\bfH{\\boldsymbol{H}}\n\\def\\bfI{\\boldsymbol{I}}\n\\def\\bfJ{\\boldsymbol{J}}\n\\def\\bfK{\\boldsymbol{K}}\n\\def\\bfL{\\boldsymbol{L}}\n\\def\\bfM{\\boldsymbol{M}}\n\\def\\bfN{\\boldsymbol{N}}\n\\def\\bfO{\\boldsymbol{O}}\n\\def\\bfP{\\boldsymbol{P}}\n\\def\\bfQ{\\boldsymbol{Q}}\n\\def\\bfR{\\boldsymbol{R}}\n\\def\\bfS{\\boldsymbol{S}}\n\\def\\bfT{\\boldsymbol{T}}\n\\def\\bfU{\\boldsymbol{U}}\n\\def\\bfV{\\boldsymbol{V}}\n\\def\\bfW{\\boldsymbol{W}}\n\\def\\bfX{\\boldsymbol{X}}\n\\def\\bfY{\\boldsymbol{Y}}\n\\def\\bfZ{\\boldsymbol{Z}}\n\\def\\bfalpha{\\boldsymbol{\\alpha}}\n\\def\\bfa{\\boldsymbol{a}}\n\\def\\bfbeta{\\boldsymbol{\\beta}}\n\\def\\bfb{\\boldsymbol{b}}\n\\def\\bfcd{\\dot{\\bfc}}\n\\def\\bfchi{\\boldsymbol{\\chi}}\n\\def\\bfc{\\boldsymbol{c}}\n\\def\\bfd{\\boldsymbol{d}}\n\\def\\bfe{\\boldsymbol{e}}\n\\def\\bff{\\boldsymbol{f}}\n\\def\\bfgamma{\\boldsymbol{\\gamma}}\n\\def\\bfg{\\boldsymbol{g}}\n\\def\\bfh{\\boldsymbol{h}}\n\\def\\bfi{\\boldsymbol{i}}\n\\def\\bfj{\\boldsymbol{j}}\n\\def\\bfk{\\boldsymbol{k}}\n\\def\\bflambda{\\boldsymbol{\\lambda}}\n\\def\\bfl{\\boldsymbol{l}}\n\\def\\bfm{\\boldsymbol{m}}\n\\def\\bfn{\\boldsymbol{n}}\n\\def\\bfomega{\\boldsymbol{\\omega}}\n\\def\\bfone{\\boldsymbol{1}}\n\\def\\bfo{\\boldsymbol{o}}\n\\def\\bfpdd{\\ddot{\\bfp}}\n\\def\\bfpd{\\dot{\\bfp}}\n\\def\\bfphi{\\boldsymbol{\\phi}}\n\\def\\bfp{\\boldsymbol{p}}\n\\def\\bfq{\\boldsymbol{q}}\n\\def\\bfr{\\boldsymbol{r}}\n\\def\\bfsigma{\\boldsymbol{\\sigma}}\n\\def\\bfs{\\boldsymbol{s}}\n\\def\\bftau{\\boldsymbol{\\tau}}\n\\def\\bftheta{\\boldsymbol{\\theta}}\n\\def\\bft{\\boldsymbol{t}}\n\\def\\bfu{\\boldsymbol{u}}\n\\def\\bfv{\\boldsymbol{v}}\n\\def\\bfw{\\boldsymbol{w}}\n\\def\\bfxi{\\boldsymbol{\\xi}}\n\\def\\bfx{\\boldsymbol{x}}\n\\def\\bfy{\\boldsymbol{y}}\n\\def\\bfzero{\\boldsymbol{0}}\n\\def\\bfz{\\boldsymbol{z}}\n\\def\\defeq{\\stackrel{\\mathrm{def}}{=}}\n\\def\\p{\\boldsymbol{p}}\n\\def\\qdd{\\ddot{\\bfq}}\n\\def\\qd{\\dot{\\bfq}}\n\\def\\q{\\boldsymbol{q}}\n\\def\\xd{\\dot{x}}\n\\def\\yd{\\dot{y}}\n\\def\\zd{\\dot{z}}\n\n2^{32} -1<\/annotation><\/semantics><\/math><\/span><span aria-hidden=\"true\" class=\"katex-html\"><span class=\"base\"><span class=\"strut\" style=\"height:0.8974em;vertical-align:-0.0833em;\"><\/span><span class=\"mord\"><span class=\"mord\">2<\/span><span class=\"msupsub\"><span class=\"vlist-t\"><span class=\"vlist-r\"><span class=\"vlist\" style=\"height:0.8141em;\"><span style=\"top:-3.063em;margin-right:0.05em;\"><span class=\"pstrut\" style=\"height:2.7em;\"><\/span><span class=\"sizing reset-size6 size3 mtight\"><span class=\"mord mtight\"><span class=\"mord mtight\">32<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><span class=\"mspace\" style=\"margin-right:0.2222em;\"><\/span><span class=\"mbin\">\u2212<\/span><span class=\"mspace\" style=\"margin-right:0.2222em;\"><\/span><\/span><span class=\"base\"><span class=\"strut\" style=\"height:0.6444em;\"><\/span><span class=\"mord\">1<\/span><\/span><\/span><\/span> entries (roughly dense matrices with dimensions larger than <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><semantics><mrow><mn>4.6<\/mn><mo>\u2217<\/mo><mn>1<\/mn><msup><mn>0<\/mn><mn>4<\/mn><\/msup><mo>\u00d7<\/mo><mn>4.6<\/mn><mo>\u2217<\/mo><mn>1<\/mn><msup><mn>0<\/mn><mn>4<\/mn><\/msup><\/mrow><annotation encoding=\"application\/x-tex\">\\def\\bfA{\\boldsymbol{A}}\n\\def\\bfB{\\boldsymbol{B}}\n\\def\\bfC{\\boldsymbol{C}}\n\\def\\bfD{\\boldsymbol{D}}\n\\def\\bfE{\\boldsymbol{E}}\n\\def\\bfF{\\boldsymbol{F}}\n\\def\\bfG{\\boldsymbol{G}}\n\\def\\bfH{\\boldsymbol{H}}\n\\def\\bfI{\\boldsymbol{I}}\n\\def\\bfJ{\\boldsymbol{J}}\n\\def\\bfK{\\boldsymbol{K}}\n\\def\\bfL{\\boldsymbol{L}}\n\\def\\bfM{\\boldsymbol{M}}\n\\def\\bfN{\\boldsymbol{N}}\n\\def\\bfO{\\boldsymbol{O}}\n\\def\\bfP{\\boldsymbol{P}}\n\\def\\bfQ{\\boldsymbol{Q}}\n\\def\\bfR{\\boldsymbol{R}}\n\\def\\bfS{\\boldsymbol{S}}\n\\def\\bfT{\\boldsymbol{T}}\n\\def\\bfU{\\boldsymbol{U}}\n\\def\\bfV{\\boldsymbol{V}}\n\\def\\bfW{\\boldsymbol{W}}\n\\def\\bfX{\\boldsymbol{X}}\n\\def\\bfY{\\boldsymbol{Y}}\n\\def\\bfZ{\\boldsymbol{Z}}\n\\def\\bfalpha{\\boldsymbol{\\alpha}}\n\\def\\bfa{\\boldsymbol{a}}\n\\def\\bfbeta{\\boldsymbol{\\beta}}\n\\def\\bfb{\\boldsymbol{b}}\n\\def\\bfcd{\\dot{\\bfc}}\n\\def\\bfchi{\\boldsymbol{\\chi}}\n\\def\\bfc{\\boldsymbol{c}}\n\\def\\bfd{\\boldsymbol{d}}\n\\def\\bfe{\\boldsymbol{e}}\n\\def\\bff{\\boldsymbol{f}}\n\\def\\bfgamma{\\boldsymbol{\\gamma}}\n\\def\\bfg{\\boldsymbol{g}}\n\\def\\bfh{\\boldsymbol{h}}\n\\def\\bfi{\\boldsymbol{i}}\n\\def\\bfj{\\boldsymbol{j}}\n\\def\\bfk{\\boldsymbol{k}}\n\\def\\bflambda{\\boldsymbol{\\lambda}}\n\\def\\bfl{\\boldsymbol{l}}\n\\def\\bfm{\\boldsymbol{m}}\n\\def\\bfn{\\boldsymbol{n}}\n\\def\\bfomega{\\boldsymbol{\\omega}}\n\\def\\bfone{\\boldsymbol{1}}\n\\def\\bfo{\\boldsymbol{o}}\n\\def\\bfpdd{\\ddot{\\bfp}}\n\\def\\bfpd{\\dot{\\bfp}}\n\\def\\bfphi{\\boldsymbol{\\phi}}\n\\def\\bfp{\\boldsymbol{p}}\n\\def\\bfq{\\boldsymbol{q}}\n\\def\\bfr{\\boldsymbol{r}}\n\\def\\bfsigma{\\boldsymbol{\\sigma}}\n\\def\\bfs{\\boldsymbol{s}}\n\\def\\bftau{\\boldsymbol{\\tau}}\n\\def\\bftheta{\\boldsymbol{\\theta}}\n\\def\\bft{\\boldsymbol{t}}\n\\def\\bfu{\\boldsymbol{u}}\n\\def\\bfv{\\boldsymbol{v}}\n\\def\\bfw{\\boldsymbol{w}}\n\\def\\bfxi{\\boldsymbol{\\xi}}\n\\def\\bfx{\\boldsymbol{x}}\n\\def\\bfy{\\boldsymbol{y}}\n\\def\\bfzero{\\boldsymbol{0}}\n\\def\\bfz{\\boldsymbol{z}}\n\\def\\defeq{\\stackrel{\\mathrm{def}}{=}}\n\\def\\p{\\boldsymbol{p}}\n\\def\\qdd{\\ddot{\\bfq}}\n\\def\\qd{\\dot{\\bfq}}\n\\def\\q{\\boldsymbol{q}}\n\\def\\xd{\\dot{x}}\n\\def\\yd{\\dot{y}}\n\\def\\zd{\\dot{z}}\n\n4.6 * 10^4 \\times 4.6 * 10^4<\/annotation><\/semantics><\/math><\/span><span aria-hidden=\"true\" class=\"katex-html\"><span class=\"base\"><span class=\"strut\" style=\"height:0.6444em;\"><\/span><span class=\"mord\">4.6<\/span><span class=\"mspace\" style=\"margin-right:0.2222em;\"><\/span><span class=\"mbin\">\u2217<\/span><span class=\"mspace\" style=\"margin-right:0.2222em;\"><\/span><\/span><span class=\"base\"><span class=\"strut\" style=\"height:0.8974em;vertical-align:-0.0833em;\"><\/span><span class=\"mord\">1<\/span><span class=\"mord\"><span class=\"mord\">0<\/span><span class=\"msupsub\"><span class=\"vlist-t\"><span class=\"vlist-r\"><span class=\"vlist\" style=\"height:0.8141em;\"><span style=\"top:-3.063em;margin-right:0.05em;\"><span class=\"pstrut\" style=\"height:2.7em;\"><\/span><span class=\"sizing reset-size6 size3 mtight\"><span class=\"mord mtight\">4<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><span class=\"mspace\" style=\"margin-right:0.2222em;\"><\/span><span class=\"mbin\">\u00d7<\/span><span class=\"mspace\" style=\"margin-right:0.2222em;\"><\/span><\/span><span class=\"base\"><span class=\"strut\" style=\"height:0.6444em;\"><\/span><span class=\"mord\">4.6<\/span><span class=\"mspace\" style=\"margin-right:0.2222em;\"><\/span><span class=\"mbin\">\u2217<\/span><span class=\"mspace\" style=\"margin-right:0.2222em;\"><\/span><\/span><span class=\"base\"><span class=\"strut\" style=\"height:0.8141em;\"><\/span><span class=\"mord\">1<\/span><span class=\"mord\"><span class=\"mord\">0<\/span><span class=\"msupsub\"><span class=\"vlist-t\"><span class=\"vlist-r\"><span class=\"vlist\" style=\"height:0.8141em;\"><span style=\"top:-3.063em;margin-right:0.05em;\"><span class=\"pstrut\" style=\"height:2.7em;\"><\/span><span class=\"sizing reset-size6 size3 mtight\"><span class=\"mord mtight\">4<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>) as it uses a BLAS version with 32-bit integers <a href=\"https:\/\/github.com\/cvxopt\/cvxopt\/issues\/126\">cvxopt#126<\/a>. According to the discussion, it seems currently impossible to avoid this, or do you know any workaround?<\/p>","category":{"@attributes":{"term":"misc"}}},{"title":"Posted by: St\u00e9phane","link":{"@attributes":{"href":"https:\/\/scaron.info\/blog\/quadratic-programming-in-python.html\/#comment-reply-large-unconstrained-dense","rel":"alternate"}},"published":"2022-12-13T10:25:00+01:00","updated":"2022-12-13T10:25:00+01:00","author":{"name":"St\u00e9phane"},"id":"tag:scaron.info,2022-12-13:\/blog\/quadratic-programming-in-python.html\/\/comment-reply-large-unconstrained-dense","summary":"<p>That's an interesting use case. I just tried the following snippet with <a class=\"reference external\" href=\"https:\/\/github.com\/qpsolvers\/qpsolvers\">qpsolvers<\/a>:<\/p>\n<pre class=\"code python literal-block\">\n<span class=\"kn\">import<\/span><span class=\"w\"> <\/span><span class=\"nn\">numpy<\/span><span class=\"w\"> <\/span><span class=\"k\">as<\/span><span class=\"w\"> <\/span><span class=\"nn\">np<\/span><span class=\"w\">\n<\/span><span class=\"kn\">from<\/span><span class=\"w\"> <\/span><span class=\"nn\">qpsolvers<\/span><span class=\"w\"> <\/span><span class=\"kn\">import<\/span> <span class=\"n\">available_solvers<\/span><span class=\"p\">,<\/span> <span class=\"n\">solve_qp<\/span><span class=\"w\">\n\n<\/span><span class=\"n\">n<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">1000<\/span>  <span class=\"c1\"># change to your liking<\/span><span class=\"w\">\n<\/span><span class=\"n\">M<\/span> <span class=\"o\">=<\/span> <span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">random<\/span><span class=\"o\">.<\/span><span class=\"n\">random<\/span><span class=\"p\">((<\/span><span class=\"n\">n<\/span><span class=\"p\">,<\/span> <span class=\"n\">n<\/span><span class=\"p\">))<\/span><span class=\"w\">\n<\/span><span class=\"n\">P<\/span> <span class=\"o\">=<\/span> <span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">dot<\/span><span class=\"p\">(<\/span><span class=\"n\">M<\/span><span class=\"o\">.<\/span><span class=\"n\">T<\/span><span class=\"p\">,<\/span> <span class=\"n\">M<\/span><span class=\"p\">)<\/span>  <span class=\"c1\"># this is a positive definite matrix<\/span><span class=\"w\">\n<\/span><span class=\"n\">q<\/span> <span class=\"o\">=<\/span> <span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">dot<\/span><span class=\"p\">(<\/span><span class=\"n\">np \u2026<\/span><\/pre>","content":"<p>That's an interesting use case. I just tried the following snippet with <a class=\"reference external\" href=\"https:\/\/github.com\/qpsolvers\/qpsolvers\">qpsolvers<\/a>:<\/p>\n<pre class=\"code python literal-block\">\n<span class=\"kn\">import<\/span><span class=\"w\"> <\/span><span class=\"nn\">numpy<\/span><span class=\"w\"> <\/span><span class=\"k\">as<\/span><span class=\"w\"> <\/span><span class=\"nn\">np<\/span><span class=\"w\">\n<\/span><span class=\"kn\">from<\/span><span class=\"w\"> <\/span><span class=\"nn\">qpsolvers<\/span><span class=\"w\"> <\/span><span class=\"kn\">import<\/span> <span class=\"n\">available_solvers<\/span><span class=\"p\">,<\/span> <span class=\"n\">solve_qp<\/span><span class=\"w\">\n\n<\/span><span class=\"n\">n<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">1000<\/span>  <span class=\"c1\"># change to your liking<\/span><span class=\"w\">\n<\/span><span class=\"n\">M<\/span> <span class=\"o\">=<\/span> <span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">random<\/span><span class=\"o\">.<\/span><span class=\"n\">random<\/span><span class=\"p\">((<\/span><span class=\"n\">n<\/span><span class=\"p\">,<\/span> <span class=\"n\">n<\/span><span class=\"p\">))<\/span><span class=\"w\">\n<\/span><span class=\"n\">P<\/span> <span class=\"o\">=<\/span> <span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">dot<\/span><span class=\"p\">(<\/span><span class=\"n\">M<\/span><span class=\"o\">.<\/span><span class=\"n\">T<\/span><span class=\"p\">,<\/span> <span class=\"n\">M<\/span><span class=\"p\">)<\/span>  <span class=\"c1\"># this is a positive definite matrix<\/span><span class=\"w\">\n<\/span><span class=\"n\">q<\/span> <span class=\"o\">=<\/span> <span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">dot<\/span><span class=\"p\">(<\/span><span class=\"n\">np<\/span><span class=\"o\">.<\/span><span class=\"n\">ones<\/span><span class=\"p\">(<\/span><span class=\"n\">n<\/span><span class=\"p\">,<\/span> <span class=\"n\">dtype<\/span><span class=\"o\">=<\/span><span class=\"nb\">float<\/span><span class=\"p\">),<\/span> <span class=\"n\">M<\/span><span class=\"p\">)<\/span><span class=\"w\">\n<\/span><span class=\"k\">for<\/span> <span class=\"n\">solver<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">available_solvers<\/span><span class=\"p\">:<\/span><span class=\"w\">\n<\/span>    <span class=\"k\">try<\/span><span class=\"p\">:<\/span><span class=\"w\">\n<\/span>        <span class=\"n\">found_solution<\/span> <span class=\"o\">=<\/span> <span class=\"n\">solve_qp<\/span><span class=\"p\">(<\/span><span class=\"n\">P<\/span><span class=\"p\">,<\/span> <span class=\"n\">q<\/span><span class=\"p\">,<\/span> <span class=\"n\">solver<\/span><span class=\"o\">=<\/span><span class=\"n\">solver<\/span><span class=\"p\">)<\/span> <span class=\"ow\">is<\/span> <span class=\"ow\">not<\/span> <span class=\"kc\">None<\/span><span class=\"w\">\n<\/span>    <span class=\"k\">except<\/span> <span class=\"ne\">Exception<\/span><span class=\"p\">:<\/span><span class=\"w\">\n<\/span>        <span class=\"n\">found_solution<\/span> <span class=\"o\">=<\/span> <span class=\"kc\">False<\/span><span class=\"w\">\n<\/span>    <span class=\"k\">if<\/span> <span class=\"n\">found_solution<\/span><span class=\"p\">:<\/span><span class=\"w\">\n<\/span>        <span class=\"nb\">print<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"s2\">&quot;Potential solver: <\/span><span class=\"si\">{<\/span><span class=\"n\">solver<\/span><span class=\"si\">}<\/span><span class=\"s2\">&quot;<\/span><span class=\"p\">)<\/span>\n<\/pre>\n<p>I have no assumption as to the conditioning of your problem, so <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><semantics><mrow><mi>M<\/mi><\/mrow><annotation encoding=\"application\/x-tex\">\\def\\bfA{\\boldsymbol{A}}\n\\def\\bfB{\\boldsymbol{B}}\n\\def\\bfC{\\boldsymbol{C}}\n\\def\\bfD{\\boldsymbol{D}}\n\\def\\bfE{\\boldsymbol{E}}\n\\def\\bfF{\\boldsymbol{F}}\n\\def\\bfG{\\boldsymbol{G}}\n\\def\\bfH{\\boldsymbol{H}}\n\\def\\bfI{\\boldsymbol{I}}\n\\def\\bfJ{\\boldsymbol{J}}\n\\def\\bfK{\\boldsymbol{K}}\n\\def\\bfL{\\boldsymbol{L}}\n\\def\\bfM{\\boldsymbol{M}}\n\\def\\bfN{\\boldsymbol{N}}\n\\def\\bfO{\\boldsymbol{O}}\n\\def\\bfP{\\boldsymbol{P}}\n\\def\\bfQ{\\boldsymbol{Q}}\n\\def\\bfR{\\boldsymbol{R}}\n\\def\\bfS{\\boldsymbol{S}}\n\\def\\bfT{\\boldsymbol{T}}\n\\def\\bfU{\\boldsymbol{U}}\n\\def\\bfV{\\boldsymbol{V}}\n\\def\\bfW{\\boldsymbol{W}}\n\\def\\bfX{\\boldsymbol{X}}\n\\def\\bfY{\\boldsymbol{Y}}\n\\def\\bfZ{\\boldsymbol{Z}}\n\\def\\bfalpha{\\boldsymbol{\\alpha}}\n\\def\\bfa{\\boldsymbol{a}}\n\\def\\bfbeta{\\boldsymbol{\\beta}}\n\\def\\bfb{\\boldsymbol{b}}\n\\def\\bfcd{\\dot{\\bfc}}\n\\def\\bfchi{\\boldsymbol{\\chi}}\n\\def\\bfc{\\boldsymbol{c}}\n\\def\\bfd{\\boldsymbol{d}}\n\\def\\bfe{\\boldsymbol{e}}\n\\def\\bff{\\boldsymbol{f}}\n\\def\\bfgamma{\\boldsymbol{\\gamma}}\n\\def\\bfg{\\boldsymbol{g}}\n\\def\\bfh{\\boldsymbol{h}}\n\\def\\bfi{\\boldsymbol{i}}\n\\def\\bfj{\\boldsymbol{j}}\n\\def\\bfk{\\boldsymbol{k}}\n\\def\\bflambda{\\boldsymbol{\\lambda}}\n\\def\\bfl{\\boldsymbol{l}}\n\\def\\bfm{\\boldsymbol{m}}\n\\def\\bfn{\\boldsymbol{n}}\n\\def\\bfomega{\\boldsymbol{\\omega}}\n\\def\\bfone{\\boldsymbol{1}}\n\\def\\bfo{\\boldsymbol{o}}\n\\def\\bfpdd{\\ddot{\\bfp}}\n\\def\\bfpd{\\dot{\\bfp}}\n\\def\\bfphi{\\boldsymbol{\\phi}}\n\\def\\bfp{\\boldsymbol{p}}\n\\def\\bfq{\\boldsymbol{q}}\n\\def\\bfr{\\boldsymbol{r}}\n\\def\\bfsigma{\\boldsymbol{\\sigma}}\n\\def\\bfs{\\boldsymbol{s}}\n\\def\\bftau{\\boldsymbol{\\tau}}\n\\def\\bftheta{\\boldsymbol{\\theta}}\n\\def\\bft{\\boldsymbol{t}}\n\\def\\bfu{\\boldsymbol{u}}\n\\def\\bfv{\\boldsymbol{v}}\n\\def\\bfw{\\boldsymbol{w}}\n\\def\\bfxi{\\boldsymbol{\\xi}}\n\\def\\bfx{\\boldsymbol{x}}\n\\def\\bfy{\\boldsymbol{y}}\n\\def\\bfzero{\\boldsymbol{0}}\n\\def\\bfz{\\boldsymbol{z}}\n\\def\\defeq{\\stackrel{\\mathrm{def}}{=}}\n\\def\\p{\\boldsymbol{p}}\n\\def\\qdd{\\ddot{\\bfq}}\n\\def\\qd{\\dot{\\bfq}}\n\\def\\q{\\boldsymbol{q}}\n\\def\\xd{\\dot{x}}\n\\def\\yd{\\dot{y}}\n\\def\\zd{\\dot{z}}\n\nM<\/annotation><\/semantics><\/math><\/span><span class=\"katex-html\" aria-hidden=\"true\"><span class=\"base\"><span class=\"strut\" style=\"height:0.6833em;\"><\/span><span class=\"mord mathnormal\" style=\"margin-right:0.10903em;\">M<\/span><\/span><\/span><\/span> is a random matrix here. Running with <code>n = 1000<\/code> gives us a first set of candidate solvers for this kind of problems: CVXOPT, ECOS, HiGHS, OSQP, ProxQP (dense backend) and quadprog. Those are only six out of the eleven available solvers.<\/p>\n<ul class=\"simple\">\n<li>At <code>n = 5000<\/code> ECOS and HiGHS don't return after one minute of runtime on my laptop, so I assume they won't scale up to your problem sizes. The other four solvers still pass.<\/li>\n<li>At <code>n = 10_000<\/code> OSQP and quadprog don't return after one minute of runtime, so I assume they won't scale either. The remaining two solvers still pass.<\/li>\n<\/ul>\n<p>This leaves us with <a class=\"reference external\" href=\"https:\/\/cvxopt.org\/\">CVXOPT<\/a> and <a class=\"reference external\" href=\"https:\/\/github.com\/Simple-Robotics\/proxsuite#proxqp\">ProxQP<\/a> as the best candidates so far. Your problem is large but it should be fit in RAM with double precision on a beefy machine (128 GB of RAM). My laptop has 8 GB of RAM, so I can only extrapolate and let you try things out by yourself from there on \ud83d\ude09<\/p>\n","category":{"@attributes":{"term":"misc"}}},{"title":"Posted by: Andreas","link":{"@attributes":{"href":"https:\/\/scaron.info\/blog\/quadratic-programming-in-python.html\/#comment-question-large-unconstrained-dense","rel":"alternate"}},"published":"2022-12-12T17:06:00-08:00","updated":"2022-12-12T17:06:00-08:00","author":{"name":"Andreas"},"id":"tag:scaron.info,2022-12-12:\/blog\/quadratic-programming-in-python.html\/\/comment-question-large-unconstrained-dense","summary":"<p>Hi, thanks very much for the interesting blog post! I am interested in large scale unconstrained quadratic optimization with positive definite, symmetric <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><semantics><mrow><mi>P<\/mi><\/mrow><annotation encoding=\"application\/x-tex\">\\def\\bfA{\\boldsymbol{A}}\n\\def\\bfB{\\boldsymbol{B}}\n\\def\\bfC{\\boldsymbol{C}}\n\\def\\bfD{\\boldsymbol{D}}\n\\def\\bfE{\\boldsymbol{E}}\n\\def\\bfF{\\boldsymbol{F}}\n\\def\\bfG{\\boldsymbol \u2026<\/annotation><\/semantics><\/math><\/span><\/span><\/p>","content":"<p>Hi, thanks very much for the interesting blog post! I am interested in large scale unconstrained quadratic optimization with positive definite, symmetric <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><semantics><mrow><mi>P<\/mi><\/mrow><annotation encoding=\"application\/x-tex\">\\def\\bfA{\\boldsymbol{A}}\n\\def\\bfB{\\boldsymbol{B}}\n\\def\\bfC{\\boldsymbol{C}}\n\\def\\bfD{\\boldsymbol{D}}\n\\def\\bfE{\\boldsymbol{E}}\n\\def\\bfF{\\boldsymbol{F}}\n\\def\\bfG{\\boldsymbol{G}}\n\\def\\bfH{\\boldsymbol{H}}\n\\def\\bfI{\\boldsymbol{I}}\n\\def\\bfJ{\\boldsymbol{J}}\n\\def\\bfK{\\boldsymbol{K}}\n\\def\\bfL{\\boldsymbol{L}}\n\\def\\bfM{\\boldsymbol{M}}\n\\def\\bfN{\\boldsymbol{N}}\n\\def\\bfO{\\boldsymbol{O}}\n\\def\\bfP{\\boldsymbol{P}}\n\\def\\bfQ{\\boldsymbol{Q}}\n\\def\\bfR{\\boldsymbol{R}}\n\\def\\bfS{\\boldsymbol{S}}\n\\def\\bfT{\\boldsymbol{T}}\n\\def\\bfU{\\boldsymbol{U}}\n\\def\\bfV{\\boldsymbol{V}}\n\\def\\bfW{\\boldsymbol{W}}\n\\def\\bfX{\\boldsymbol{X}}\n\\def\\bfY{\\boldsymbol{Y}}\n\\def\\bfZ{\\boldsymbol{Z}}\n\\def\\bfalpha{\\boldsymbol{\\alpha}}\n\\def\\bfa{\\boldsymbol{a}}\n\\def\\bfbeta{\\boldsymbol{\\beta}}\n\\def\\bfb{\\boldsymbol{b}}\n\\def\\bfcd{\\dot{\\bfc}}\n\\def\\bfchi{\\boldsymbol{\\chi}}\n\\def\\bfc{\\boldsymbol{c}}\n\\def\\bfd{\\boldsymbol{d}}\n\\def\\bfe{\\boldsymbol{e}}\n\\def\\bff{\\boldsymbol{f}}\n\\def\\bfgamma{\\boldsymbol{\\gamma}}\n\\def\\bfg{\\boldsymbol{g}}\n\\def\\bfh{\\boldsymbol{h}}\n\\def\\bfi{\\boldsymbol{i}}\n\\def\\bfj{\\boldsymbol{j}}\n\\def\\bfk{\\boldsymbol{k}}\n\\def\\bflambda{\\boldsymbol{\\lambda}}\n\\def\\bfl{\\boldsymbol{l}}\n\\def\\bfm{\\boldsymbol{m}}\n\\def\\bfn{\\boldsymbol{n}}\n\\def\\bfomega{\\boldsymbol{\\omega}}\n\\def\\bfone{\\boldsymbol{1}}\n\\def\\bfo{\\boldsymbol{o}}\n\\def\\bfpdd{\\ddot{\\bfp}}\n\\def\\bfpd{\\dot{\\bfp}}\n\\def\\bfphi{\\boldsymbol{\\phi}}\n\\def\\bfp{\\boldsymbol{p}}\n\\def\\bfq{\\boldsymbol{q}}\n\\def\\bfr{\\boldsymbol{r}}\n\\def\\bfsigma{\\boldsymbol{\\sigma}}\n\\def\\bfs{\\boldsymbol{s}}\n\\def\\bftau{\\boldsymbol{\\tau}}\n\\def\\bftheta{\\boldsymbol{\\theta}}\n\\def\\bft{\\boldsymbol{t}}\n\\def\\bfu{\\boldsymbol{u}}\n\\def\\bfv{\\boldsymbol{v}}\n\\def\\bfw{\\boldsymbol{w}}\n\\def\\bfxi{\\boldsymbol{\\xi}}\n\\def\\bfx{\\boldsymbol{x}}\n\\def\\bfy{\\boldsymbol{y}}\n\\def\\bfzero{\\boldsymbol{0}}\n\\def\\bfz{\\boldsymbol{z}}\n\\def\\defeq{\\stackrel{\\mathrm{def}}{=}}\n\\def\\p{\\boldsymbol{p}}\n\\def\\qdd{\\ddot{\\bfq}}\n\\def\\qd{\\dot{\\bfq}}\n\\def\\q{\\boldsymbol{q}}\n\\def\\xd{\\dot{x}}\n\\def\\yd{\\dot{y}}\n\\def\\zd{\\dot{z}}\n\nP<\/annotation><\/semantics><\/math><\/span><span aria-hidden=\"true\" class=\"katex-html\"><span class=\"base\"><span class=\"strut\" style=\"height:0.6833em;\"><\/span><span class=\"mord mathnormal\" style=\"margin-right:0.13889em;\">P<\/span><\/span><\/span><\/span> (dense, with sizes up <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><semantics><mrow><mn>1<\/mn><msup><mn>0<\/mn><mn>5<\/mn><\/msup><mo>\u00d7<\/mo><mn>1<\/mn><msup><mn>0<\/mn><mn>5<\/mn><\/msup><\/mrow><annotation encoding=\"application\/x-tex\">\\def\\bfA{\\boldsymbol{A}}\n\\def\\bfB{\\boldsymbol{B}}\n\\def\\bfC{\\boldsymbol{C}}\n\\def\\bfD{\\boldsymbol{D}}\n\\def\\bfE{\\boldsymbol{E}}\n\\def\\bfF{\\boldsymbol{F}}\n\\def\\bfG{\\boldsymbol{G}}\n\\def\\bfH{\\boldsymbol{H}}\n\\def\\bfI{\\boldsymbol{I}}\n\\def\\bfJ{\\boldsymbol{J}}\n\\def\\bfK{\\boldsymbol{K}}\n\\def\\bfL{\\boldsymbol{L}}\n\\def\\bfM{\\boldsymbol{M}}\n\\def\\bfN{\\boldsymbol{N}}\n\\def\\bfO{\\boldsymbol{O}}\n\\def\\bfP{\\boldsymbol{P}}\n\\def\\bfQ{\\boldsymbol{Q}}\n\\def\\bfR{\\boldsymbol{R}}\n\\def\\bfS{\\boldsymbol{S}}\n\\def\\bfT{\\boldsymbol{T}}\n\\def\\bfU{\\boldsymbol{U}}\n\\def\\bfV{\\boldsymbol{V}}\n\\def\\bfW{\\boldsymbol{W}}\n\\def\\bfX{\\boldsymbol{X}}\n\\def\\bfY{\\boldsymbol{Y}}\n\\def\\bfZ{\\boldsymbol{Z}}\n\\def\\bfalpha{\\boldsymbol{\\alpha}}\n\\def\\bfa{\\boldsymbol{a}}\n\\def\\bfbeta{\\boldsymbol{\\beta}}\n\\def\\bfb{\\boldsymbol{b}}\n\\def\\bfcd{\\dot{\\bfc}}\n\\def\\bfchi{\\boldsymbol{\\chi}}\n\\def\\bfc{\\boldsymbol{c}}\n\\def\\bfd{\\boldsymbol{d}}\n\\def\\bfe{\\boldsymbol{e}}\n\\def\\bff{\\boldsymbol{f}}\n\\def\\bfgamma{\\boldsymbol{\\gamma}}\n\\def\\bfg{\\boldsymbol{g}}\n\\def\\bfh{\\boldsymbol{h}}\n\\def\\bfi{\\boldsymbol{i}}\n\\def\\bfj{\\boldsymbol{j}}\n\\def\\bfk{\\boldsymbol{k}}\n\\def\\bflambda{\\boldsymbol{\\lambda}}\n\\def\\bfl{\\boldsymbol{l}}\n\\def\\bfm{\\boldsymbol{m}}\n\\def\\bfn{\\boldsymbol{n}}\n\\def\\bfomega{\\boldsymbol{\\omega}}\n\\def\\bfone{\\boldsymbol{1}}\n\\def\\bfo{\\boldsymbol{o}}\n\\def\\bfpdd{\\ddot{\\bfp}}\n\\def\\bfpd{\\dot{\\bfp}}\n\\def\\bfphi{\\boldsymbol{\\phi}}\n\\def\\bfp{\\boldsymbol{p}}\n\\def\\bfq{\\boldsymbol{q}}\n\\def\\bfr{\\boldsymbol{r}}\n\\def\\bfsigma{\\boldsymbol{\\sigma}}\n\\def\\bfs{\\boldsymbol{s}}\n\\def\\bftau{\\boldsymbol{\\tau}}\n\\def\\bftheta{\\boldsymbol{\\theta}}\n\\def\\bft{\\boldsymbol{t}}\n\\def\\bfu{\\boldsymbol{u}}\n\\def\\bfv{\\boldsymbol{v}}\n\\def\\bfw{\\boldsymbol{w}}\n\\def\\bfxi{\\boldsymbol{\\xi}}\n\\def\\bfx{\\boldsymbol{x}}\n\\def\\bfy{\\boldsymbol{y}}\n\\def\\bfzero{\\boldsymbol{0}}\n\\def\\bfz{\\boldsymbol{z}}\n\\def\\defeq{\\stackrel{\\mathrm{def}}{=}}\n\\def\\p{\\boldsymbol{p}}\n\\def\\qdd{\\ddot{\\bfq}}\n\\def\\qd{\\dot{\\bfq}}\n\\def\\q{\\boldsymbol{q}}\n\\def\\xd{\\dot{x}}\n\\def\\yd{\\dot{y}}\n\\def\\zd{\\dot{z}}\n\n10^5 \\times 10^5<\/annotation><\/semantics><\/math><\/span><span aria-hidden=\"true\" class=\"katex-html\"><span class=\"base\"><span class=\"strut\" style=\"height:0.8974em;vertical-align:-0.0833em;\"><\/span><span class=\"mord\">1<\/span><span class=\"mord\"><span class=\"mord\">0<\/span><span class=\"msupsub\"><span class=\"vlist-t\"><span class=\"vlist-r\"><span class=\"vlist\" style=\"height:0.8141em;\"><span style=\"top:-3.063em;margin-right:0.05em;\"><span class=\"pstrut\" style=\"height:2.7em;\"><\/span><span class=\"sizing reset-size6 size3 mtight\"><span class=\"mord mtight\">5<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><span class=\"mspace\" style=\"margin-right:0.2222em;\"><\/span><span class=\"mbin\">\u00d7<\/span><span class=\"mspace\" style=\"margin-right:0.2222em;\"><\/span><\/span><span class=\"base\"><span class=\"strut\" style=\"height:0.8141em;\"><\/span><span class=\"mord\">1<\/span><span class=\"mord\"><span class=\"mord\">0<\/span><span class=\"msupsub\"><span class=\"vlist-t\"><span class=\"vlist-r\"><span class=\"vlist\" style=\"height:0.8141em;\"><span style=\"top:-3.063em;margin-right:0.05em;\"><span class=\"pstrut\" style=\"height:2.7em;\"><\/span><span class=\"sizing reset-size6 size3 mtight\"><span class=\"mord mtight\">5<\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span><\/span>). Are the presented solvers still usable?<\/p>","category":{"@attributes":{"term":"misc"}}},{"title":"Posted by: TJ","link":{"@attributes":{"href":"https:\/\/scaron.info\/blog\/quadratic-programming-in-python.html\/#comment-question-cvxopt-equality","rel":"alternate"}},"published":"2022-05-09T10:46:00+08:00","updated":"2022-05-09T10:46:00+08:00","author":{"name":"TJ"},"id":"tag:scaron.info,2022-05-09:\/blog\/quadratic-programming-in-python.html\/\/comment-question-cvxopt-equality","content":"<p>The wrapped function <code>cvxopt_solve_qp<\/code> is probably wrong, what if we only have equality constraint? That code won't work.<\/p>","category":{"@attributes":{"term":"misc"}}},{"title":"Posted by: St\u00e9phane","link":{"@attributes":{"href":"https:\/\/scaron.info\/blog\/quadratic-programming-in-python.html\/#comment-reply-cvxopt-equality","rel":"alternate"}},"published":"2021-06-27T14:42:00+02:00","updated":"2021-06-27T14:42:00+02:00","author":{"name":"St\u00e9phane"},"id":"tag:scaron.info,2021-06-27:\/blog\/quadratic-programming-in-python.html\/\/comment-reply-cvxopt-equality","summary":"<p>Thank you for pointing this out. Handling all cases is a bit verbose and not\nadding to the points made in this post, so I've updated the inline code to\nassume clearly that <tt class=\"docutils literal\">G<\/tt> and <tt class=\"docutils literal\">h<\/tt> are set. For a general solution, all cases\nare handled in <a class=\"reference external\" href=\"https:\/\/github.com\/qpsolvers\/qpsolvers\">qpsolvers<\/a>, which\nstarted \u2026<\/p>","content":"<p>Thank you for pointing this out. Handling all cases is a bit verbose and not\nadding to the points made in this post, so I've updated the inline code to\nassume clearly that <tt class=\"docutils literal\">G<\/tt> and <tt class=\"docutils literal\">h<\/tt> are set. For a general solution, all cases\nare handled in <a class=\"reference external\" href=\"https:\/\/github.com\/qpsolvers\/qpsolvers\">qpsolvers<\/a>, which\nstarted from this blog post but has evolved to include fixes, features (such as\nbox inequalities) and new solvers.<\/p>\n","category":{"@attributes":{"term":"misc"}}}]}