30 September 2017

Fix slow container shutdown

Tags: Docker

Today I was working in a dev environment where everything was done within containers and docker-compose was used to define the services. Usually, when I work on my own projects in this way I don't have to rebuild containers every time I make changes to my code to reflect those changes but in this special case, there was no getting around it. Every time I made changes to the code I had to rebuild the container and run it again. The annoying part was that it took such a long time to re-up the container because the stop process took such a long time. I investigated it and it seems I have been running my apps in the container as the PID 1 process. How does this happen?

There are two ways of running your containers. With CMD or ENTRYPOINT or a combination of both. For example

 

FROM alpine:3.6

CMD ["ping"]

and

FROM alpine:3.6
ENTRYPOINT ["ping"]
CMD ["8.8.8.8"]

are two ways of defining how your container should later be run. 

CMD lets the user that runs the container overwrite the command where ENTRYPOINT does not give this freedom thus making the container a ping only app.

In both cases running the containers, the ping application is run as PID 1 and comes with a bunch of responsibilities and edge cases that we have not anticipated - notably the handling of SIGTERMs.

Lets have a look at the container

docker run --rm -it --name pinger donchev7/alpine-pinger
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=59 time=10.643 ms
64 bytes from 8.8.8.8: seq=1 ttl=59 time=9.861 ms
64 bytes from 8.8.8.8: seq=2 ttl=59 time=12.448 ms

The donchev7/alpine-pinger image is only 3.97MB, you should have no problem running this container in seconds.

While the container is running issue this command in another terminal window

docker exec pinger ps aux

you should see the following

docker exec pinger ps aux                               
PID   USER     TIME   COMMAND
1     root     0:00   ping 8.8.8.8
7     root     0:00   ps aux

There it is, ping is PID 1

Connecting the dots

When we issue a docker stop command the docker daemon sends a SIGTERM signal to stop our pinger but since it's PID 1 nothing happens and after a while (10s) the docker daemon loses its patience with the container and sends a SIGKILL. Why does this happen? Obviously, because our application running in the container wasn't programmed to listen to SIGTERM.
While still having your pinger container running run the following

time docker stop pinger

On my computer it took 10,577 seconds to stop the container. That's crazy, containers should be lighting fast :)

Solutions

  • Add STOPSIGNAL SIGINT  to your Dockerfile (this is a quick & dirty solution and only takes care of one edge case and shouldn't actually be used

  • Avoid being PID 1 when you run ad-hoc docker run commands you can pass the--init flagand your app won't become PID 1

Let's test the second solution out. Run the pinger again this time passing the --init flag like so

docker run --rm -it --name pinger --init donchev7/alpine-pinger

and then

time docker stop pinger

Now it takes around 500ms to stop the running container!

BONUS

If you are using docker-compose and a YML version 2.2 or 2.3 you can do the same

version: '2.3'
services:
  pinger:
    image: donchev7/alpine-pinger
    init: true

save this in a docker-compose.yml file and your docker-compose stop command should be lightning fast!
 

 

 

 

 

18  question_answer

  • said on 15 July 2018 19:06

    To reach your goals it's essential that you read-through and recognize every theme regarding a certain game.

  • said on 11 April 2019 23:39

    Wonderful article! This is the kind of info that are supposed to be shared across the internet. Disgrace on Google for not positioning this publish higher! Come on over and seek advice from my web site . Thanks =)

  • said on 06 June 2019 02:24

    In this age people share videos so many times a day they need a high quality and safe app to perform this task. Xender is one of them. With this app you can transfer your images everywhere you want, from your Android to iOS and vice versa. You may download Xender app from <a href=https://xender-download.com>https://xender-download.com</a> for free!

  • said on 24 July 2019 01:20

    Excellent goods from you, man. I have understand your stuff previous to and you are just too wonderful. I really like what you've acquired here, certainly like what you're saying and the way in which you say it. You make it entertaining and you still take care of to keep it sensible. I can't wait to read much more from you. This is really a tremendous web site.

  • said on 27 July 2019 08:56

    Wow, this post is pleasant, my younger sister is analyzing these things, therefore I am going to convey her.

  • said on 14 September 2019 23:59

    Thanks for the marvelous posting! I seriously enjoyed reading it, you can be a great author.I wilpl always bookmark your blog and will eventually come back from now on. I want to encourage you continue your great job, have a nice holiday weekend!

  • said on 16 October 2019 08:55

    Hi Dear, are you in fact visiting this web site regularly, if so after that you will without doubt take fastidious knowledge.

  • said on 16 October 2019 17:36

    I'm gone to say to my little brother, that he should aso pay a visit tgis web site on regular basis to obtain updated from hottest reports.

  • said on 22 October 2019 23:22

    My partner and I stumbled over here coming from a different page and thought I might aas well check things out. I like whbat I see so now i'm following you. Look forward too looking over your wweb page repeatedly.

  • said on 23 November 2019 00:58

    Hi, i feel that i saw you visited my blog so i came to return the prefer?.I'm trying to find issues to enhance my web site!I suppose its ok to make use of a few of yohr ideas!!

  • said on 01 December 2019 19:38

    I don't ecen know how I ended up here, buut I assumed this post used to be great. I don't recognize who you might be however certainly you aare going to a well-known blogger if you are not already. Cheers!

  • said on 13 March 2020 20:40

    400 Mg Viagra <a href=http://apcialisle.com/#>cialis</a> Buy Generic Viagra Using Paypal <a href=http://apcialisle.com/#>Cialis</a> Mixing Amoxicillin Wine Together

  • said on 14 March 2020 06:53

    Viagra Otras Alternativas <a href=http://apcialisle.com/#>Buy Cialis</a> 5151.1?Esetprotoscanctx Ca31208 <a href=http://apcialisle.com/#>Buy Cialis</a> Viagra Effet Photo

  • said on 10 January 2020 10:10

    You coould definitely see your skills in thhe woprk you write. The arena hopes for more passionate writers such as yoou who aren't afraid to say how they believe. Always go after your heart.

  • said on 08 March 2020 02:56

    Buy Letrozole Gynecomastia <a href=https://apcialisle.com/#>Buy Cialis</a> Propecia Cost Per Month <a href=https://apcialisle.com/#>Buy Cialis</a> Women Taking Propecia Treatment

  • said on 16 March 2020 03:13

    Retin A Cream <a href=http://apcialisle.com/#>viagra vs cialis</a> Zithromax Before Surgery <a href=http://apcialisle.com/#>generic cialis canada</a> Buy Zithromax Online No Prescription Fast Delivery

  • said on 18 March 2020 11:12

    Acheter Du Cialis <a href=https://apcialisle.com/#>cheapest cialis</a> Usa Billed Viagra <a href=https://apcialisle.com/#>Cialis</a> Propecia Oferta Farmacia

  • said on 23 March 2020 13:28

    Viagra Generika Cialis <a href=https://apcialisle.com/#>buy generic cialis</a> Acheter Du Vrai Propecia Sur Internet <a href=https://apcialisle.com/#>Cialis</a> Pharmacy Online Usa

Leave a comment

Your email address will not be published. It's only used for gravatars