Skip to content

Let apt process finish before running another#13287

Closed
matt-kwong wants to merge 1 commit intogrpc:masterfrom
matt-kwong:kokoro-lock
Closed

Let apt process finish before running another#13287
matt-kwong wants to merge 1 commit intogrpc:masterfrom
matt-kwong:kokoro-lock

Conversation

@matt-kwong
Copy link
Copy Markdown
Contributor

@matt-kwong matt-kwong commented Nov 7, 2017 •
edited
Loading

Trying to reproduce this bug

+ sudo apt-get install -y jq
E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?

Reproduced while checking for an apt process: https://sponge.corp.google.com/target?id=0c2cdf6b-1c76-43cc-aff4-08b8f88092b9&target=grpc%2Fubuntu%2Fpull_request%2Fgrpc_basictests_cpp_opt&searchFor=

+ ps -A
+ grep apt
 3513 ?        00:00:00 apt.systemd.dai
 3525 ?        00:00:00 apt.systemd.dai
+ sudo apt-get install -y jq
E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?

Looks like newer versions of Ubunutu have a daemon that updates system packages daily and on boot. This update locks apt-get causing apt-get install to fail in our builds.

Possible solution: wait for all apt processes to finish before using apt-get. Will test this PR a few times to make sure that we can expect the daily apt daemon to finish.

@grpc-testing
Copy link
Copy Markdown

grpc-testing commented Nov 7, 2017 

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]


****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

@grpc-testing
Copy link
Copy Markdown

grpc-testing commented Nov 7, 2017 

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]


****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

@grpc-testing
Copy link
Copy Markdown

grpc-testing commented Nov 7, 2017 

[trickle] No significant performance differences

1 similar comment
@grpc-testing
Copy link
Copy Markdown

grpc-testing commented Nov 7, 2017 

[trickle] No significant performance differences

@grpc-testing
Copy link
Copy Markdown

grpc-testing commented Nov 7, 2017 

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]


****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

@matt-kwong matt-kwong changed the title Test if apt process is locking Let apt process finish before running another Nov 7, 2017
@grpc-testing
Copy link
Copy Markdown

grpc-testing commented Nov 7, 2017 

[trickle] No significant performance differences

@grpc-testing
Copy link
Copy Markdown

grpc-testing commented Nov 7, 2017 

[microbenchmarks] No significant performance differences

2 similar comments
@grpc-testing
Copy link
Copy Markdown

grpc-testing commented Nov 7, 2017 

[microbenchmarks] No significant performance differences

@grpc-testing
Copy link
Copy Markdown

grpc-testing commented Nov 7, 2017 

[microbenchmarks] No significant performance differences

@jtattermusch jtattermusch mentioned this pull request Nov 7, 2017
Merged
jtattermusch
jtattermusch approved these changes Nov 7, 2017
View reviewed changes
Copy link
Copy Markdown
Contributor

@jtattermusch jtattermusch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but can't we also just disable the package updates?

@matt-kwong
Copy link
Copy Markdown
Contributor Author

matt-kwong commented Nov 7, 2017

It looks like this issue is specific to our own images. I'll disable the daemon in our image instead of this PR.

@matt-kwong
Copy link
Copy Markdown
Contributor Author

matt-kwong commented Nov 8, 2017

I updated the images to disable the daemon and ran 3 rounds of tests. None of them failed with the lock issue. This is likely a resolved issue.

@matt-kwong matt-kwong closed this Nov 8, 2017
@lock lock bot locked as resolved and limited conversation to collaborators Jan 21, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer

@jtattermusch jtattermusch jtattermusch approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@jtattermusch jtattermusch

Labels

cla: yes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@matt-kwong @grpc-testing @jtattermusch @googlebot @kokoro-team