Skip to content

Add utility methods to dump catalogs to JSON - #441

Closed
cdeil wants to merge 3 commits into
gammapy:masterfrom
cdeil:cat-to-json
Closed

cdeil wants to merge 3 commits into
gammapy:masterfrom
cdeil:cat-to-json

Conversation

@cdeil

@cdeil cdeil commented Feb 8, 2016

Copy link
Copy Markdown
Contributor

This pull request implements utility methods to dump catalogs to JSON.
This is useful for displaying catalog information in web pages.

@cdeil cdeil added the feature label Feb 8, 2016
@cdeil cdeil self-assigned this Feb 8, 2016
@cdeil cdeil added this to the 0.4 milestone Feb 8, 2016
@cdeil cdeil assigned adonath and unassigned cdeil Feb 11, 2016
@cdeil

cdeil commented Feb 11, 2016

Copy link
Copy Markdown
Contributor Author

@adonath - I've assigned this issue to you and given you write access to my fork.
Thanks for implementing this!

@cdeil

cdeil commented Feb 12, 2016

Copy link
Copy Markdown
Contributor Author

I now think trying to implement our own Table JSON serialisation is pointless / futile ... sorry @adonath for leading you down this path. See astropy/astropy#4604

I'll sleep on this and wait for feedback on astropy/astropy#4604 and then do something here ... probably drop vector columns for now and then call table.to_pandas().to_json().

@kepta - If you want to start with the website displaying 2FHL catalog info already, you could do something like this, i.e. just keep a few columns and it'll work:

from astropy.table import Table
table = Table.read('https://github.com/gammapy/gammapy-extra/blob/master/datasets/catalogs/fermi/gll_psch_v08.fit.gz?raw=true')
cols = ['Source_Name', 'RAJ2000', 'DEJ2000', 'GLON', 'GLAT', 'CLASS']
df = table[cols].to_pandas()
df.to_json()

Let me know if you have any questions!

@adonath

adonath commented Feb 15, 2016

Copy link
Copy Markdown
Member

@cdeil No problem, using pandas dataframes seems to be the better solution...close PR?

@cdeil

cdeil commented Feb 15, 2016

Copy link
Copy Markdown
Contributor Author

@adonath - If you have time, could you add to_pandas methods that remove the array-valued columns or re-format them to multiple single-valued columns?

Having this would be useful e.g. when building this webpage:
https://github.com/gammapy/web-experiments/pull/6
or also the HGPS webpage...

@cdeil

cdeil commented Feb 16, 2016

Copy link
Copy Markdown
Contributor Author

Here's a proposal for a solution:
astropy/astropy#4604 (comment)

The key is to get rid of the Numpy data types by calling to_list. A Python int is JSON serialisable, but a numpy.int64 object isn't ... that's what making the tests fail here.

@adonath - Do you have time to do this or should I?

@cdeil cdeil assigned cdeil and unassigned adonath Feb 19, 2016
@cdeil cdeil modified the milestones: never, 0.4 Feb 19, 2016
@cdeil

cdeil commented Feb 19, 2016

Copy link
Copy Markdown
Contributor Author

I'm closing this PR now.

For now, we can dump the catalog info we want with a few lines of Python code:
astropy/astropy#4604 (comment)
http://nbviewer.jupyter.org/gist/cdeil/5c1a31ae00b3a0eefe84

So it's probably not needed that we introduce JSON dump code in Gammapy.

@cdeil cdeil closed this Feb 19, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants