Skip to content

Conversation

@philip-peterson
Copy link
Contributor

When encountering an HTTP 413 error, a message like the following is displayed:

LFS: Client error &{%!!(string=http) %!!(string=) %!!(*url.Userinfo=<nil>) %!!(string=my.example.com) %!!(string=/git/redacted/redcated.git/info/lfs/objects/9c4bf89ecc9faef1abab8a0c27a349a34b30b6d5816c5352dc26e0ceded0963f/1119460) %!!(string=) %!!(bool=false) %!!(bool=false) %!!(string=) %!!(string=) %!!(string=)}s(MISSING) from HTTP 413

This patch changes it to something readable:

LFS: Client error http://my.example.com/git/redacted/redacted.git/info/lfs/objects/9c4bf89ecc9faef1abab8a0c27a349a34b30b6d5816c5352dc26e0ceded0963f/1119460 from HTTP 413

@philip-peterson philip-peterson requested a review from a team as a code owner July 10, 2024 08:28
@chrisd8088
Copy link
Member

Hey, thanks for the contribution!

This looks logical, at first glance, but it seems it does cause problems somehow; some of our CI tests are reporting things like:

Authorization error: %!s(MISSING)
Check that you have proper access to the repository%!!(MISSING)(EXTRA *url.URL=http://127.0.0.1:35771/storage/invalid4c48d2a6991c9895bcddcf027e1e4907280bcf21975492b1afbade396d6a3340?r=push-invalid-pushinsteadof)

So we'll have to investigate a bit more and figure out what the intended action is here in the different use cases.

Watchara242540

This comment was marked as spam.

chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Feb 28, 2025
In PR git-lfs#4781 we updated a number of packages to replace global string
variables with local variables so they could be translated via the
"tr" package's localization functions.

One concern raised in that PR pertained to the changes to the
lfshttp/errors.go source file, where a map of error format strings
is constructed in the defaultError() function, with each string
mapped to a specific HTTP status code.  These strings should contain
at least one "%s" verb, into which the HTTP request URL should be
interpolated.

In commit cda9c3c of PR git-lfs#4781 we moved
the map's initialization into the defaultError() function of our
"lfshttp" package, and added a call to the Get() method of the Tr
structure from our "tr" package around each string's definition.
However, we deferred the interpolation of the request URL to a
subsequent call to a string formatting function, with the intent that
we would first select the correct localized format string from the
map based on the HTTP request's status code, and then interpolate
the request's URL into the final error message.

Further, if a request's status code does not correspond to a string
in the map, our defaultError() function creates a generic error
message into which we try to interpolate both the request's URL and
its status code.  For these generic messages, the intent of the
changes in commit cda9c3c was that
we first interpolate the status code, and then treat the resultant
string as if it had been found in the defaultErrors map, meaning it
should still contain one "%s" verb into which the request URL can be
interpolated.

Such a design implies that in the generic error messages we need to
use a "%d" verb for the status code, and a doubly-escaped "%%s" verb
for the request URL, since we want the latter to be converted to a
regular "%s" verb when the string is first used as a format string.

This design exposed an unexpected issue, which is described in
leonelquinteros/gotext#57.  In trying to work around this issue, though,
we introduced a bug of our own whereby the generic messages are passed
through three formatting functions rather than two, and in doing so
the "%%s" verbs are eliminated before we attempt to interpolate the
HTTP request's URL, leading to further errors.

The single commit in PR git-lfs#5822 attempts to fix this problem, but without
complete success.  We now adopt the changes from that PR's commit and
resolve the remaining problems so we should always supply an appropriate
set of arguments for the number of formatting verbs in both types of
messages in the defaultError() function.

The confusion here stems from the issue reported in
leonelquinteros/gotext#57, namely that the Printf() function of the
"github.com/leonelquinteros/gotext" package behaves differently
depending on whether or not it is passed any arguments other than
a format string.  When no arguments are passed, the format string
is not parsed and any verbs are left unaltered, whereas when
at least one additional argument is passed, the format string is
parsed and the trailing arguments interpolated into it.

The Tr global variable in our "tr" package is a Locale structure
from the "gotext" package.  Starting with commit
cda9c3c, we separately invoke this
variable's Get() method on each of the error messages defined in
the defaultErrors map, but without any other arguments.  Because
the Get() method does not parse the format string when no other
arguments are passed, in theory we ought to be able to define these
messages with regular "%s" verbs.  If we do so, though, the Go linter
complains that it appears we are passing format strings to the Get()
method without matching arguments for their "%s" verbs.

To work around this linter limitation, we currently use "%%s" verbs in
the error messages in the defaultErrors map, which are then passed
through the Get() method unchanged.  We then make a subsequent call
to the Sprintf() function from the Go standard library's "fmt" package,
passing the error message as the format string but without any other
arguments.  This call effectively just converts any "%%s" verbs to
"%s" verbs.

By contrast, the generic error messages in the defaultError() function
are passed to the Tr variable's Get() method with another argument,
the HTTP request's status code.  Due to the presence of the extra
argument, the Get() method parses the format string, replacing the "%d"
verb with the status code and converting the "%%s" verb into a "%s"
verb.  (Note that originally, in PR git-lfs#4781, we used the Sprintf()
function of the standard "fmt" package rather than the Get() method
of our Tr variable.  We corrected this oversight in commit
04abbd8 of PR git-lfs#4846, but while that
change allows the strings to be localized, it has no effect on the
handling of the formatting verbs.)

As with the error messages from the defaultErrors map, the generic
error messages are then also passed without any other arguments to the
Sprintf() function from the standard "fmt" package.  However, this call
does not have the same effect in both cases.  When the error message is
specific to an HTTP status code and has been selected from the
defaultErrors map, this call just converts the "%%s" verb into a "%s"
one, as noted above.  When the error message is a generic one, though,
that conversion has already been performed by the Tr structure's Get()
method.  The consequence is that the Sprintf() function replaces the
"%s" verb with "%!s(MISSING)", as described in the "fmt" package's
documentation:

  https://pkg.go.dev/fmt#hdr-Format_errors

In the final step of the defaultError() function we then pass the
result from this Sprintf() call to the Errof() function of our
"errors" package, and here we include the HTTP request's URL as a second
argument.  In the case where the error message has been selected from
the defaultErrors map, this last formatting function replaces the "%s"
verb with the URL.  In the case where the error message is a generic one,
though, the leading portion of the format error marker "%!s(MISSING)"
cannot be parsed as a verb, and so the "%!" is replaced with a full
reflection of all the internal elements of the URL structure into
string form.

Since these format error markers and reflection fields are not desirable
in our HTTP request error messages, we revise the defaultError() function
so it each type of error message is passed through only a single
formatting function, specifically the Tr variable's Get() method.
This allows us to use regular "%s" and "%d" formatting verbs exclusively
in all the messages.

Rather than call the Get() method on each string as it is defined in
the defaultErrors map, we only call the method once a specific message
is selected based on the HTTP request's status code, and we pass the
request's URL to the Get() method as another argument.  For the generic
error messages, we simply pass both the status code and the URL as
arguments to the Get() function.  Because we always pass either one or
two arguments to that function in addition to a format string, it always
performs the appropriate interpolation for us, and we avoid the need
to work around its behaviour when it is only passed a format string
without other arguments, as well as avoiding the use of any further
formatting functions like Sprintf().

Co-authored-by: Philip Peterson <[email protected]>
@chrisd8088
Copy link
Member

Thanks again for this PR, @philip-peterson!

I've included your changes in PR #5998 along with some other revisions that help resolve the outstanding CI failures from this PR, and credited you on the relevant commit.

I believe that PR should fully resolve the problems you reported here, and so I'm going to close this PR. Thank you again for identifying the bug and helping us to fix it!

@chrisd8088 chrisd8088 closed this Feb 28, 2025
@philip-peterson
Copy link
Contributor Author

Great, thank you @chrisd8088 and thanks for maintaining this futuristic and forward-looking extension to Git.

chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Mar 5, 2025
In PR git-lfs#4781 we updated a number of packages to replace global string
variables with local variables so they could be translated via the
"tr" package's localization functions.

One concern raised in that PR pertained to the changes to the
lfshttp/errors.go source file, where a map of error format strings
is constructed in the defaultError() function, with each string
mapped to a specific HTTP status code.  These strings should contain
at least one "%s" verb, into which the HTTP request URL should be
interpolated.

In commit cda9c3c of PR git-lfs#4781 we moved
the map's initialization into the defaultError() function of our
"lfshttp" package, and added a call to the Get() method of the Tr
structure from our "tr" package around each string's definition.
However, we deferred the interpolation of the request URL to a
subsequent call to a string formatting function, with the intent that
we would first select the correct localized format string from the
map based on the HTTP request's status code, and then interpolate
the request's URL into the final error message.

Further, if a request's status code does not correspond to a string
in the map, our defaultError() function creates a generic error
message into which we try to interpolate both the request's URL and
its status code.  For these generic messages, the intent of the
changes in commit cda9c3c was that
we first interpolate the status code, and then treat the resultant
string as if it had been found in the defaultErrors map, meaning it
should still contain one "%s" verb into which the request URL can be
interpolated.

Such a design implies that in the generic error messages we need to
use a "%d" verb for the status code, and a doubly-escaped "%%s" verb
for the request URL, since we want the latter to be converted to a
regular "%s" verb when the string is first used as a format string.

This design exposed an unexpected issue, which is described in
leonelquinteros/gotext#57.  In trying to work around this issue, though,
we introduced a bug of our own whereby the generic messages are passed
through three formatting functions rather than two, and in doing so
the "%%s" verbs are eliminated before we attempt to interpolate the
HTTP request's URL, leading to further errors.

The single commit in PR git-lfs#5822 attempts to fix this problem, but without
complete success.  We now adopt the changes from that PR's commit and
resolve the remaining problems so we should always supply an appropriate
set of arguments for the number of formatting verbs in both types of
messages in the defaultError() function.

The confusion here stems from the issue reported in
leonelquinteros/gotext#57, namely that the Printf() function of the
"github.com/leonelquinteros/gotext" package behaves differently
depending on whether or not it is passed any arguments other than
a format string.  When no arguments are passed, the format string
is not parsed and any verbs are left unaltered, whereas when
at least one additional argument is passed, the format string is
parsed and the trailing arguments interpolated into it.

The Tr global variable in our "tr" package is a Locale structure
from the "gotext" package.  Starting with commit
cda9c3c, we separately invoke this
variable's Get() method on each of the error messages defined in
the defaultErrors map, but without any other arguments.  Because
the Get() method does not parse the format string when no other
arguments are passed, in theory we ought to be able to define these
messages with regular "%s" verbs.  If we do so, though, the Go linter
complains that it appears we are passing format strings to the Get()
method without matching arguments for their "%s" verbs.

To work around this linter limitation, we currently use "%%s" verbs in
the error messages in the defaultErrors map, which are then passed
through the Get() method unchanged.  We then make a subsequent call
to the Sprintf() function from the Go standard library's "fmt" package,
passing the error message as the format string but without any other
arguments.  This call effectively just converts any "%%s" verbs to
"%s" verbs.

By contrast, the generic error messages in the defaultError() function
are passed to the Tr variable's Get() method with another argument,
the HTTP request's status code.  Due to the presence of the extra
argument, the Get() method parses the format string, replacing the "%d"
verb with the status code and converting the "%%s" verb into a "%s"
verb.  (Note that originally, in PR git-lfs#4781, we used the Sprintf()
function of the standard "fmt" package rather than the Get() method
of our Tr variable.  We corrected this oversight in commit
04abbd8 of PR git-lfs#4846, but while that
change allows the strings to be localized, it has no effect on the
handling of the formatting verbs.)

As with the error messages from the defaultErrors map, the generic
error messages are then also passed without any other arguments to the
Sprintf() function from the standard "fmt" package.  However, this call
does not have the same effect in both cases.  When the error message is
specific to an HTTP status code and has been selected from the
defaultErrors map, this call just converts the "%%s" verb into a "%s"
one, as noted above.  When the error message is a generic one, though,
that conversion has already been performed by the Tr structure's Get()
method.  The consequence is that the Sprintf() function replaces the
"%s" verb with "%!s(MISSING)", as described in the "fmt" package's
documentation:

  https://pkg.go.dev/fmt#hdr-Format_errors

In the final step of the defaultError() function we then pass the
result from this Sprintf() call to the Errof() function of our
"errors" package, and here we include the HTTP request's URL as a second
argument.  In the case where the error message has been selected from
the defaultErrors map, this last formatting function replaces the "%s"
verb with the URL.  In the case where the error message is a generic one,
though, the leading portion of the format error marker "%!s(MISSING)"
cannot be parsed as a verb, and so the "%!" is replaced with a full
reflection of all the internal elements of the URL structure into
string form.

Since these format error markers and reflection fields are not desirable
in our HTTP request error messages, we revise the defaultError() function
so it each type of error message is passed through only a single
formatting function, specifically the Tr variable's Get() method.
This allows us to use regular "%s" and "%d" formatting verbs exclusively
in all the messages.

Rather than call the Get() method on each string as it is defined in
the defaultErrors map, we only call the method once a specific message
is selected based on the HTTP request's status code, and we pass the
request's URL to the Get() method as another argument.  For the generic
error messages, we simply pass both the status code and the URL as
arguments to the Get() function.  Because we always pass either one or
two arguments to that function in addition to a format string, it always
performs the appropriate interpolation for us, and we avoid the need
to work around its behaviour when it is only passed a format string
without other arguments, as well as avoiding the use of any further
formatting functions like Sprintf().

Co-authored-by: Philip Peterson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants