Skip to content

gene_cnv_frequencies PerformanceWarning: DataFrame is highly fragmented #67

@cclarkson

Description

@cclarkson

We get four warnings about fragmented pandas dataframes when we use the cohort sets (due to how many columns we are building I think) - would be good to fix the code to avoid these.

/home/conda/store/4b0d6587ea35727c87000368f18c95bf1e775a25ab9791007f1cca148f9a452c-binder-v3.2.0/lib/python3.8/site-packages/malariagen_data/ag3.py:1755: PerformanceWarning: DataFrame is highly fragmented. This is usually the result of calling frame.insertmany times, which has poor performance. Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, usenewframe = frame.copy()df[f"{coh}_amp"] = amp_freq_coh /home/conda/store/4b0d6587ea35727c87000368f18c95bf1e775a25ab9791007f1cca148f9a452c-binder-v3.2.0/lib/python3.8/site-packages/malariagen_data/ag3.py:1756: PerformanceWarning: DataFrame is highly fragmented. This is usually the result of callingframe.insertmany times, which has poor performance. Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, usenewframe = frame.copy()df[f"{coh}_del"] = del_freq_coh /home/conda/store/4b0d6587ea35727c87000368f18c95bf1e775a25ab9791007f1cca148f9a452c-binder-v3.2.0/lib/python3.8/site-packages/malariagen_data/ag3.py:1746: PerformanceWarning: DataFrame is highly fragmented. This is usually the result of callingframe.insertmany times, which has poor performance. Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, usenewframe = frame.copy()df[f"{coh}_amp"] = np.nan /home/conda/store/4b0d6587ea35727c87000368f18c95bf1e775a25ab9791007f1cca148f9a452c-binder-v3.2.0/lib/python3.8/site-packages/malariagen_data/ag3.py:1747: PerformanceWarning: DataFrame is highly fragmented. This is usually the result of callingframe.insertmany times, which has poor performance. Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, usenewframe = frame.copy() df[f"{coh}_del"] = np.nan

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions