Skip to content

Fast histogram observer #29790

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
raghuramank100 wants to merge 10 commits into from
Closed

Fast histogram observer #29790

raghuramank100 wants to merge 10 commits into from

Conversation

@raghuramank100
Copy link
Contributor

@raghuramank100 raghuramank100 commented Nov 14, 2019 •
edited
Loading

Stack from ghstack:

10x speed up of histogram observers by rewriting histogram combination routine.
Previously this was done as explicit bilinear interpolation.
Now this is done as a sample rate conversion operation, where we achieve
resampling by an upsampling (zero-order hold) followed by box filtering and
downsampling.

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
X = torch.randn(10,1,320,320)
start = time.time()
obs(X)
#obs.forward_new(X)
acc_time = acc_time + time.time()-start
print(acc_time)

Before change:
6.9

After change:
0.6

Resnet-18 accuracy is unchanged with the faster histogram observer.
Acc1 = 69.4

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D18508562

@raghuramank10000
Fast histogram observer
2f2c566 
Summary:

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
raghuramank100 pushed a commit that referenced this pull request Nov 14, 2019
@raghuramank10000
Fast histogram observer
6e9754b 
Summary:

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: c6c795d
Pull Request resolved: #29790
@raghuramank10000
Update on "Fast histogram observer"
d234fe7 
Summary:

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@raghuramank100 raghuramank100 mentioned this pull request Nov 14, 2019
Closed
@raghuramank10000
Update on "Fast histogram observer"
cb14d12 
Summary:

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
raghuramank100 pushed a commit that referenced this pull request Nov 14, 2019
@raghuramank10000
Fast histogram observer
d35317a 
Summary:

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 87a1583
Pull Request resolved: #29790
@raghuramank10000
Update on "Fast histogram observer"
25949fe 
Summary:

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D18508562](https://our.internmc.facebook.com/intern/diff/D18508562)

[ghstack-poisoned]
@raghuramank10000
Update on "Fast histogram observer"
5e30ea5 
Summary:

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D18508562](https://our.internmc.facebook.com/intern/diff/D18508562)

[ghstack-poisoned]
raghuramank100 pushed a commit that referenced this pull request Nov 14, 2019
@raghuramank10000
Fast histogram observer
f61c7c2 
Summary:

 Speed up histogram observers by rewriting histogram combination routine.
 Previously this was done as explicit bilinear interpolation.
 Now this is done as a sample rate conversion operation, where we achieve
 resampling by an upsampling (zero-order hold) followed by box filtering and
 downsampling.

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Before change:
6.9

After change:
0.6

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 09481b9
Pull Request resolved: #29790
@raghuramank10000
Update on "Fast histogram observer"
8170df9 
Summary:

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D18508562](https://our.internmc.facebook.com/intern/diff/D18508562)

[ghstack-poisoned]
@raghuramank10000
Update on "Fast histogram observer"
fcd4ad7 
Summary:

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D18508562](https://our.internmc.facebook.com/intern/diff/D18508562)

[ghstack-poisoned]
raghuramank100 pushed a commit that referenced this pull request Nov 15, 2019
@raghuramank10000
Fast histogram observer
8de5ea6 
Summary:

 Speed up histogram observers by rewriting histogram combination routine.
 Previously this was done as explicit bilinear interpolation.
 Now this is done as a sample rate conversion operation, where we achieve
 resampling by an upsampling (zero-order hold) followed by box filtering and
 downsampling.

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Before change:
6.9

After change:
0.6

Resnet-18 accuracy is unchanged with the faster histogram observer.
Acc1 = 69.4

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 7b2778e
Pull Request resolved: #29790
lly-zero-one
lly-zero-one reviewed Nov 15, 2019
View reviewed changes
Copy link
Contributor

@lly-zero-one lly-zero-one left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NVM, it seems you already added it.

hx89
hx89 reviewed Nov 19, 2019
View reviewed changes
Copy link

@hx89 hx89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great idea and huge speedup!

torch/quantization/observer.py Outdated
dtype=torch.double)[downsample_rate - 1 :: downsample_rate]
# Finally perform interpolation
shifted_integral_histogram = torch.zeros((Nbins))
shifted_integral_histogram[0] = 0
Copy link

@hx89 hx89 Nov 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Seems this is not needed since it's initialized to zero in line 739?

torch/quantization/observer.py Outdated
self.min_val = None
self.max_val = None
self.dst_nbins = 2 ** torch.iinfo(self.dtype).bits
self.upsample_rate = 128
Copy link

@hx89 hx89 Nov 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May add comment on why choose 128 here? And could make upsample_rate as an input argument.

@raghuramank10000
Update on "Fast histogram observer"
f1529e4 
Summary:

 10x speed up of histogram observers by rewriting histogram combination routine.
 Previously this was done as explicit bilinear interpolation.
 Now this is done as a sample rate conversion operation, where we achieve
 resampling by an upsampling (zero-order hold) followed by box filtering and
 downsampling.

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Before change:
6.9

After change:
0.6

Resnet-18 accuracy is unchanged with the faster histogram observer.
Acc1 = 69.4

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D18508562](https://our.internmc.facebook.com/intern/diff/D18508562)

[ghstack-poisoned]
@raghuramank10000
Update on "Fast histogram observer"
62ef5cd 
Summary:

 10x speed up of histogram observers by rewriting histogram combination routine.
 Previously this was done as explicit bilinear interpolation.
 Now this is done as a sample rate conversion operation, where we achieve
 resampling by an upsampling (zero-order hold) followed by box filtering and
 downsampling.

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Before change:
6.9

After change:
0.6

Resnet-18 accuracy is unchanged with the faster histogram observer.
Acc1 = 69.4

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D18508562](https://our.internmc.facebook.com/intern/diff/D18508562)

[ghstack-poisoned]
raghuramank100 pushed a commit that referenced this pull request Nov 19, 2019
@raghuramank10000
Fast histogram observer
5621814 
Summary:
 Address review comments and typos

 Speed up histogram observers by rewriting histogram combination routine.
 Previously this was done as explicit bilinear interpolation.
 Now this is done as a sample rate conversion operation, where we achieve
 resampling by an upsampling (zero-order hold) followed by box filtering and
 downsampling.

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Before change:
6.9

After change:
0.6

Resnet-18 accuracy is unchanged with the faster histogram observer.
Acc1 = 69.4

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 0785394
Pull Request resolved: #29790
hx89
hx89 approved these changes Nov 19, 2019
View reviewed changes
Copy link

@hx89 hx89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@raghuramank10000
Update on "Fast histogram observer"
275a44c 
Summary:

 10x speed up of histogram observers by rewriting histogram combination routine.
 Previously this was done as explicit bilinear interpolation.
 Now this is done as a sample rate conversion operation, where we achieve
 resampling by an upsampling (zero-order hold) followed by box filtering and
 downsampling.

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Before change:
6.9

After change:
0.6

Resnet-18 accuracy is unchanged with the faster histogram observer.
Acc1 = 69.4

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D18508562](https://our.internmc.facebook.com/intern/diff/D18508562)

[ghstack-poisoned]
raghuramank100 pushed a commit that referenced this pull request Nov 19, 2019
@raghuramank10000
Fast histogram observer
5fcd9d4 
Summary:
 Address review comments and typos

 Speed up histogram observers by rewriting histogram combination routine.
 Previously this was done as explicit bilinear interpolation.
 Now this is done as a sample rate conversion operation, where we achieve
 resampling by an upsampling (zero-order hold) followed by box filtering and
 downsampling.

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Before change:
6.9

After change:
0.6

Resnet-18 accuracy is unchanged with the faster histogram observer.
Acc1 = 69.4

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 34dec38
Pull Request resolved: #29790
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Nov 20, 2019

This pull request has been merged in 67b77af.

@facebook-github-bot facebook-github-bot deleted the gh/raghuraman_k/8/head branch November 24, 2019 15:16
xxtEchjovs44 pushed a commit to xxtEchjovs44/pytorch that referenced this pull request Jan 29, 2020
@raghuramank10000
Fast histogram observer
220efcb 
Summary:

Test Plan:
import torch
import time
import numpy as np
from torch.quantization.observer import HistogramObserver

X = torch.randn(1,1,224,224)

obs = HistogramObserver(2048)
acc_time = 0
for i in range(100):
   X = torch.randn(10,1,320,320)
   start = time.time()
   obs(X)
   #obs.forward_new(X)
   acc_time = acc_time + time.time()-start
print(acc_time)

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 1a6c6f9
Pull Request resolved: pytorch/pytorch#29790
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@lly-zero-one lly-zero-one lly-zero-one left review comments

@hx89 hx89 hx89 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Merged

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@raghuramank100 @facebook-github-bot @lly-zero-one @hx89 @mruberry @raghuramank10000