Prometheusで監視・モニタリング #hackt

Transcript

1. PrometheusͰ ؂ࢹɾϞχλϦϯά Hacker Tackle 2016 2016/09/10 @matsumana

2. ࣗݾ঺հ • ໊લɿ দ࡚ ֶ • Twitterɿ @matsumana • ৬ۀɿ

   ιϑτ΢ΣΞΤϯδχΞ
 ʢϓϩάϥϚɺWebαʔϏεӡ༻ʣ

3. ΞδΣϯμ • Prometheusͷ঺հ • ଞͷπʔϧͱͷൺֱ • promgenͷ঺հ • exporterʹ͍ͭͯ

4. Ξϯέʔτ

5. ݱࡏԿΛ࢖ͬͯ·͔͢ʁ • Nagios • Ganglia • Munin • Cacti •

   Zabbix • AWS CloudWatch • Datadog • Mackerel • New Relic • ͦΕҎ֎

6. PrometheusͷࣄΛ ஌ͬͯ·͔ͨ͠ʁ

7. Prometheusͱ͸ • SoundCloud͕ࣾ։ൃ͠ɺOSSͱͯ͠ެ։͍ͯ͠Δ؂ࢹɾϞχλϦϯάπʔϧ • ࠷ॳͷcommit͸2012/11/24 • 2016/07/18ʹ1.0.0ϦϦʔε
 (ࠓ೔࣌఺Ͱͷ࠷৽͸1.1.0) • ࠷ۙ஫໨౓্͕ঢத

   • Prometheus Casual Talks @౦ژ → ࢀՃऀ100໊ • Prometheus Core Developer Björn Rabenstein Open Q&A @ϩϯυϯ
 ˠ ࢀՃऀ 27໊ • PromCon 2016 @ϕϧϦϯ → ࢀՃऀ 80໊

8. Prometheusͷػೳ • time series database (࣌ܥྻσʔλϕʔε) • ϝτϦΫεऩू & อଘ

   • อଘͨ͠ϝτϦΫεʹର͢ΔΫΤϦ • ͖͍͠஋Λઃఆͯ͠ΞϥʔτΛඈ͹͢ • μογϡϘʔυ

9. Prometheusʹແ͍ػೳ • ੜϩά/Πϕϯτσʔλͷऩू
 (Elasticsearchͷ༷ͳ) • ϦΫΤετͷτϨʔγϯά
 (Zipkinͷ༷ͳ) • ػցֶश΍σʔλϚΠχϯάʹΑΔݕग़ •

   ϝτϦΫεͷ௕ظอଘετϨʔδ • ࣗಈతͳਫฏεέʔϦϯά • Ϣʔβೝূ

10. ͦͷଞͷಛ௃ • golangͰॻ͔Ε͍ͯΔ • pullܕ
 ؂ࢹର৅ϊʔυʹexportorΛΠϯετʔϧͯ͠Prometheus͔ΒϙʔϦϯά • Grafanaͱ࿈ܞՄೳ • PrometheusࣗମʹPromDashͱ͍͏μογϡϘʔυπʔϧ͕͋Δ͕ɺ

    Grafanaͱͷ࿈ܞ͕ਪ঑ • CNCF(Cloud Native Computing Foundation)ͷϓϩδΣΫτʹͳͬͨ • Kubernetesͱ౷߹Մೳ
 ※ͱ͸ݴͬͯ΋Docker༻Ͱ͸ͳ͍ͷͰɺ΋ͪΖΜΦϯϓϨͰ΋࢖͑Δ

11. ΞʔΩςΫνϟ

12. pros • Grafanaͱ࿈ܞग़དྷΔ • ΫΤϦ͕ॻ͚Δ • αʔόεϖοΫ΍ϝτϦΫε਺ʹΑΔͱࢥ͏͕
 1୆ͷαʔόͰ୔ࢁͷϊʔυΛ؂ࢹͰ͖ΔΒ͍͠ • 1୆ͷαʔόͰ3,000ϊʔυ͘Β͍؂ࢹग़དྷΔΒ͍͠

    • AWSͷ৔߹ɺ50ʙ100ϊʔυ͘Β͍ͳΒαʔόεϖοΫ͸ t2.medium(CPU: 2Core, Mem: 4GB)Ͱे෼Β͍͠ • exportor͕๛෋ɻ΋͠ཉ͍͠΋ͷ͕ແ͚Ε͹؆୯ʹ࡞ΕΔ

13. cons • ໊લ͕ΞϨͳͷͰάάΓʹ͍͘
 ʢөըɺΪϦγϟਆ࿩ͷਆʣ • ΫΤϦͷॻ͖ํ͕Α͘Θ͔Βͳ͍ • ؂ࢹର৅ϊʔυͷ௥Ճ͸ઃఆϑΝΠϧʹॻ͍ͯ
 ϓϩηεΛ࠶ىಈʢGUI͕ͳ͍ʣ •

    TimeZone͕GMT (GrafanaΛ࢖͑͹໰୊ͳ͍)

14. ଞͷπʔϧͱͷൺֱ • ྫ͑͹Zabbixͱൺֱ͢ΔͱɺͲΜͳײ͡ʁ • γϯϓϧ ʘ(^o^)ʗ • ίϯϙʔωϯτ͕ૄ݁߹ͳͷ͸ɺ
 1ͭͷαʔϏεͰ਺ϊʔυ؂ࢹ͢Δ͚ͩͳΒZabbixͷ༷ͳ
 ΦʔϧΠϯϫϯͷํ͕͍͍͔΋

    (´ɾωɾʆ) • WebUIͰઃఆग़དྷͳ͍ (´ɾωɾʆ) • ϝϯςφϯε͕࣌ؒઃఆग़དྷͳ͍ (´ɾωɾʆ) • Ξϥʔτૹ৴ઌϝʔϧΞυϨεͷઃఆ͕ࡉ͔͘ग़དྷͳ͍ (´ɾωɾʆ)

15. Prometheus
 ྑ͍Μ͚ͩͲ ͦͷ··࢖͏ͷ͸ ݁ߏͭΒ͍

16. ͦ͜Ͱ

17. promgen • GitHubͰެ։த • https://github.com/line/promgen • WebUIͰઃఆ͕Մೳ

18. promgen Service

19. promgen Project

20. ΞʔΩςΫνϟ (࠶ܝ)

21. Prometheus w/ promgen

22. exporter

23. Α͘࢖͍ͦ͏ͳexporter • Default port allocations
 Wiki͸ࣗ༝ʹฤूՄೳɻศརͳexporterΛ࡞ͬͨΒొ࿥͠Α͏ʂ • node_exporter • blackbox_exporter

    • nginx_exporter • apache_exporter • mysqld_exporter • redis_exporter • memcached_exporter • jmx_exporter • zookeeper_exporter • elasticsearch_exporter

24. exporterͷ࡞Γํ • ΫϥΠΞϯτϥΠϒϥϦΛ࢖͏ͱָ • client_java • client_golang • client_python •

    client_ruby

25. exporterͷಈ࡞Πϝʔδ

26. exporterΛ؂ࢹର৅ΞϓϦʹ૊ΈࠐΉࣄ΋Մೳ

27. γϯϓϧͳ exporterͷιʔεΛྫʹ ࡞ΓํΛઆ໌͠·͢

28. σϞ • promgen • Prometheus • Grafana

29. ϦϯΫू • ΦϑΟγϟϧ • prometheus.io • Github Organization • YouTubeνϟϯωϧ

    ʢPromConͷಈը͋Γʣ ← ͓͢͢Ί • Prometheus Casual Talks #1Λ։࠵͠·ͨ͠ • Prometheus London meetupͰLT͖ͯ͠·ͨ͠ • PromCon 2016ʹߦͬͯൃද͖ͯ͠·ͨ͠ • promgen - prometheus managemnet tool / simpleclient_java hacks @ Prometheus casual