Skip to content

Add URL with fragment to ClientResponse#2925

Merged
asvetlov merged 5 commits intoaio-libs:masterfrom
hdk5:fix_response_url_fragment
Apr 17, 2018
Merged

Add URL with fragment to ClientResponse#2925
asvetlov merged 5 commits intoaio-libs:masterfrom
hdk5:fix_response_url_fragment

Conversation

@hdk5
Copy link
Copy Markdown
Contributor

@hdk5 hdk5 commented Apr 10, 2018 •
edited
Loading

What do these changes do?

ClientResponse and RequestInfo now have real_url property, which is request url without fragment part being stripped

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt
  • Add a new news fragment into the CHANGES folder

@hdk5
Copy link
Copy Markdown
Contributor Author

hdk5 commented Apr 10, 2018

I had a trouble retrieving URL fragment from response URL. I am aware that request should be sent without URL fragment, but I don't think that URL from response should be modified.

@codecov-io
Copy link
Copy Markdown

codecov-io commented Apr 10, 2018 •
edited
Loading

Codecov Report

Merging #2925 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2925      +/-   ##
==========================================
+ Coverage   97.99%   97.99%   +<.01%     
==========================================
  Files          40       40              
  Lines        7514     7520       +6     
  Branches     1318     1318              
==========================================
+ Hits         7363     7369       +6     
  Misses         48       48              
  Partials      103      103
Impacted Files Coverage Δ
aiohttp/client_reqrep.py 97.27% <100%> (+0.02%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 544716c...aac2dc4. Read the comment docs.

@asvetlov
Copy link
Copy Markdown
Member

asvetlov commented Apr 10, 2018

HTTP response has no URL (unless it is not 3xx redirection).
ClientResponse.url is an URL of request.
I think that keeping a fragment part in response:

  1. Makes inconsistency mess.
  2. Breaks existing API

@hdk5
Copy link
Copy Markdown
Contributor Author

hdk5 commented Apr 10, 2018

unless it is not 3xx redirection

This is what i meant. I need to know full URL I am getting redirected to because this is how I get application token.

Here is a little example: https://dry-meadow-80563.herokuapp.com/tokenask

It redirects to /tokengive#token=hereisyourtoken, and now it is not possible to get #token=hereisyourtoken part from response object.

hdk5
hdk5 commented Apr 10, 2018
View reviewed changes
aiohttp/client_reqrep.py
q.extend(url2.query)
url = url.with_query(q)
self.url = url.with_fragment(None)
self.original_url = url
Copy link
Copy Markdown
Contributor Author

@hdk5 hdk5 Apr 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this code before my commit looks strange to me:

        self.url = url.with_fragment(None)
        self.original_url = url

Both url and original_url are not modified after (at least by the object itself), so they are always the same.

And then original_url is passed as url for ClientResponse.

So, wasn't it actually intended to keep URL fragment in response object?

@asvetlov
Copy link
Copy Markdown
Member

asvetlov commented Apr 11, 2018

Let's consider ClientRequest as private API. I don't care too much about its attributes.
The library needs a public client request object but it is another story.

The public API is ClientResponse and resp.history for looking on redirection URLs.
The history is a list of RequestInfo data structures.
I propose adding a new attribute called real_url (unstripped_url or whatever) to save URL with a fragment.

By this, we will keep the behavior of existing code and provide a way for getting original unmodified URLs.

Opinions?

@hdk5
Copy link
Copy Markdown
Contributor Author

hdk5 commented Apr 11, 2018

I propose adding a new attribute called real_url

This might be a solution.

But can you please explain my previous comment on code and this commit?

@asvetlov
Copy link
Copy Markdown
Member

asvetlov commented Apr 11, 2018

Looks like @fafhrd91 wanted to implement exactly your initial intention, later we broke the behavior again.
I see two reasons for it:

  1. Logical complexity: there is no obvious answer to the question: what is response's URL. I have a feeling that we need two properties.
  2. The mentioned commit has no tests/documentation update, that's why we did break it insensibly.

@hdk5
Copy link
Copy Markdown
Contributor Author

hdk5 commented Apr 11, 2018

Should I just put real url in real url attribute then, like you suggested, and update docs?

@asvetlov
Copy link
Copy Markdown
Member

asvetlov commented Apr 11, 2018

I think we need both ClientResponse.real_url and RequestInfo.real_url.

Documentation and tests are mandatory.

@hdk5 hdk5 changed the title Fixed ClientResponse URL fragment Add URL with fragment to ClientResponse Apr 12, 2018
asvetlov
asvetlov requested changes Apr 17, 2018
View reviewed changes
docs/client_reference.rst
.. attribute:: real_url

Unmodified URL of request (:class:`~yarl.URL`).

Copy link
Copy Markdown
Member

@asvetlov asvetlov Apr 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add .. versionadded:: 3.2 tag

docs/client_reference.rst
.. attribute:: real_url

Requested *url* with URL fragment unstripped, :class:`yarl.URL` instance.

Copy link
Copy Markdown
Member

@asvetlov asvetlov Apr 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.. versionadded:: 3.2

@asvetlov asvetlov merged commit cb17696 into aio-libs:master Apr 17, 2018
@asvetlov
Copy link
Copy Markdown
Member

asvetlov commented Apr 17, 2018

Thanks!

thehesiod pushed a commit to thehesiod-forks/aiohttp that referenced this pull request Apr 26, 2018
@hdk5 @thehesiod
Add URL with fragment to ClientResponse (aio-libs#2925)
6d6f5a5 
* Fixed ClientResponse URL fragment

* Added news entry

* ClientResponse real_url property + doc + test

* Fixed flake8 and doc-spelling errors

* Updated docs
@lock
Copy link
Copy Markdown

lock bot commented Oct 28, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new

@lock lock bot added the outdated label Oct 28, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Oct 28, 2019
@psf-chronographer psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Oct 28, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@asvetlov asvetlov asvetlov approved these changes

Assignees

No one assigned

Labels

bot:chronographer:provided There is a change note present in this PR outdated

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@hdk5 @codecov-io @asvetlov