Skip to content

DeepLab: Fix generation of tfrecord for Cityscapes #8866

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
aquariusjay merged 3 commits into from
Jul 14, 2020
Merged

DeepLab: Fix generation of tfrecord for Cityscapes #8866

aquariusjay merged 3 commits into from
Jul 14, 2020

Conversation

@ruslo
Copy link
Contributor

@ruslo ruslo commented Jul 14, 2020 •
edited
Loading

Description

Fix generation of tfrecord for Cityscapes and update documentation to use the correct split --train_split="train_fine"/--eval_split="val_fine".

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Tests

python deeplab/train.py \
    --logtostderr \
    --training_number_of_steps=100 \
    --train_split="train_fine" \
    --model_variant="xception_65" \
    --fine_tune_batch_norm=False \
    --atrous_rates=6 \
    --atrous_rates=12 \
    --atrous_rates=18 \
    --output_stride=16 \
    --decoder_output_stride=4 \
    --train_crop_size="769,769" \
    --train_batch_size=1 \
    --dataset="cityscapes" \
    --tf_initial_checkpoint=${PATH_TO_INITIAL_CHECKPOINT} \
    --train_logdir=${PATH_TO_TRAIN_DIR} \
    --dataset_dir=${PATH_TO_DATASET}

python deeplab/eval.py \
    --logtostderr \
    --eval_split="val_fine" \
    --model_variant="xception_65" \
    --atrous_rates=6 \
    --atrous_rates=12 \
    --atrous_rates=18 \
    --output_stride=16 \
    --decoder_output_stride=4 \
    --eval_crop_size="1025,2049" \
    --dataset="cityscapes" \
    --max_number_of_evaluations=1 \
    --checkpoint_dir=${PATH_TO_CHECKPOINT} \
    --eval_logdir=${PATH_TO_EVAL_DIR} \
    --dataset_dir=${PATH_TO_DATASET}

python deeplab/vis.py \
    --logtostderr \
    --vis_split="val_fine" \
    --model_variant="xception_65" \
    --atrous_rates=6 \
    --atrous_rates=12 \
    --atrous_rates=18 \
    --output_stride=16 \
    --decoder_output_stride=4 \
    --vis_crop_size="1025,2049" \
    --dataset="cityscapes" \
    --max_number_of_iterations=1 \
    --colormap_type="cityscapes" \
    --checkpoint_dir=${PATH_TO_CHECKPOINT} \
    --vis_logdir=${PATH_TO_VIS_DIR} \
    --dataset_dir=${PATH_TO_DATASET}
  • Ubuntu 18.04.4
  • CUDA 10.2
  • Tensorflow 1.15.2

Checklist

@ruslo ruslo requested review from YknZhu and aquariusjay as code owners July 14, 2020 20:26
aquariusjay
aquariusjay approved these changes Jul 14, 2020
View reviewed changes
Copy link
Contributor

@aquariusjay aquariusjay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, ruslo!

@aquariusjay aquariusjay merged commit b860406 into tensorflow:master Jul 14, 2020
aquariusjay added a commit that referenced this pull request Jul 14, 2020
@aquariusjay
Revert "DeepLab: Fix generation of tfrecord for Cityscapes (#8866)"
553cd0e 
This reverts commit b860406.
@ruslo
Copy link
Contributor Author

ruslo commented Jul 14, 2020

Thanks, ruslo!

👌

@aquariusjay Can you please review/merge the #8534 too

@ruslo ruslo deleted the pr.city branch July 14, 2020 21:10
@aquariusjay
Copy link
Contributor

aquariusjay commented Jul 14, 2020

Hi ruslo,

Thanks for the changes.
I just notice one thing that we could make the code better. Could you please have another pull request for the following changes?

  1. Update the line in the main function:
    for dataset_split in ['train', 'val']:
    to
    for dataset_split in ['train_fine', 'val_fine']:

  2. Revert back your change in this line:
    shard_filename = '%s_fine-%05d-of-%05d.tfrecord'
    to
    hard_filename = '%s-%05d-of-%05d.tfrecord' % (

By doing so, it will be more consistent with the renaming splits changes you have.

What do you think?

Thanks!

@ruslo
Copy link
Contributor Author

ruslo commented Jul 14, 2020

I just notice one thing that we could make the code better. Could you please have another pull request for the following changes?

Ok, I'll check it and test it tomorrow.

@aquariusjay
Copy link
Contributor

aquariusjay commented Jul 14, 2020

Great! Thanks, ruslo!

@ruslo ruslo mentioned this pull request Jul 15, 2020
Merged
9 tasks
@ruslo
Copy link
Contributor Author

ruslo commented Jul 15, 2020

Could you please have another pull request for the following changes?

@aquariusjay See #8870

@aquariusjay
Copy link
Contributor

aquariusjay commented Jul 15, 2020

Checking it. Thanks, ruslo!

kmindspark pushed a commit to kmindspark/models that referenced this pull request Jul 19, 2020
@ruslo @kmindspark
DeepLab: Fix generation of tfrecord for Cityscapes (tensorflow#8866)
8b1c71b 
* convert_cityscapes.sh: Update PYTHONPATH

Fix error `ModuleNotFoundError: No module named 'cityscapesscripts'`

* Check the number of labels and images

* DeepLab: Fix generation of tfrecord for Cityscapes
kmindspark pushed a commit to kmindspark/models that referenced this pull request Jul 23, 2020
@ruslo @kmindspark
DeepLab: Fix generation of tfrecord for Cityscapes (tensorflow#8866)
141b341 
* convert_cityscapes.sh: Update PYTHONPATH

Fix error `ModuleNotFoundError: No module named 'cityscapesscripts'`

* Check the number of labels and images

* DeepLab: Fix generation of tfrecord for Cityscapes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@YknZhu YknZhu Awaiting requested review from YknZhu

1 more reviewer

@aquariusjay aquariusjay aquariusjay approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cla: yes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ruslo @aquariusjay @googlebot