{"id":3669,"date":"2022-08-08T20:11:47","date_gmt":"2022-08-08T12:11:47","guid":{"rendered":"http:\/\/www.chenlianfu.com\/?p=3669"},"modified":"2022-08-09T17:17:05","modified_gmt":"2022-08-09T09:17:05","slug":"%e6%8f%90%e5%8d%87ceph%e7%b3%bb%e7%bb%9f%e5%af%b9pg%e7%9a%84scrub%e5%92%8cbackfilling%e9%80%9f%e5%ba%a6","status":"publish","type":"post","link":"http:\/\/www.chenlianfu.com\/?p=3669","title":{"rendered":"\u63d0\u5347CEPH\u7cfb\u7edf\u5bf9PG\u7684Scrub\u548cBackfilling\u901f\u5ea6"},"content":{"rendered":"\n<h2>1. \u8fdb\u884cPG Srub\u7684\u53c2\u6570\u548c\u539f\u7406<\/h2>\n\n\n\n<p>\u5bf9PG\u8fdb\u884cScrub\uff0c\u5373\u68c0\u6d4bPG\u5206\u5e03\u5230\u5404OSDs\u4e0a\u7684\u6570\u636e\u662f\u5426\u4e00\u81f4\u3002\u5305\u542b\u4e24\u79cd\u7b97\u6cd5\uff1a\u7b2c\u4e00\u79cd\u4e3aScrub\uff0c\u5373\u4ec5\u68c0\u6d4b\u6570\u636e\u7684\u5143\u4fe1\u606f\uff0c\u4f8b\u5982\u65f6\u95f4\u6233\u548c\u6587\u4ef6\u5927\u5c0f\u7b49\u4fe1\u606f\uff0c\u901f\u5ea6\u5feb\u4e14\u57fa\u672c\u4e0d\u6d88\u8017\u78c1\u76d8\uff1b\u7b2c\u4e8c\u79cd\u4e3adeep Scrub\uff0c\u4f1a\u989d\u5916\u68c0\u6d4b\u6570\u636e\u7684\u5185\u5bb9\u662f\u5426\u4e00\u81f4\uff0c\u901f\u5ea6\u8f83\u6162\u4e14\u6d88\u8017\u5927\u91cf\u78c1\u76d8\u8bfb\u53d6\u3002\u5728CEPH\u7cfb\u7edf\u8fdb\u884cscrub\u548cdeep-scrub\u7684\u4e3b\u8981\u53c2\u6570\u4e3a\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\uff081\uff09osd_scrub_begin_hour\uff1a\u8bbe\u7f6e\u8fdb\u884cScrub\u7684\u8d77\u59cb\u65f6\u95f4\uff0c\u63a8\u8350\u8bbe\u7f6e\u4e3a22\uff0c\u5373\u5728\u665a\u4e0a10\u70b9\u5f00\u59cb\u8fdb\u884cScrub\u3002CEPH\u7cfb\u7edf\u9ed8\u8ba4\u503c\u662f0\u3002\n\uff082\uff09osd_scrub_end_hour\uff1a\u8bbe\u7f6e\u8fdb\u884cScrub\u7684\u7ed3\u675f\u65f6\u95f4\uff0c\u63a8\u8350\u8bbe\u7f6e\u4e3a8\uff0c\u5373\u5728\u65e9\u4e0a8\u70b9\u7ed3\u675fscrub\u3002CEPH\u7cfb\u7edf\u9ed8\u8ba4\u503c\u662f24\u3002\n\uff083\uff09osd_scrub_auto_repair\uff1a\u8bbe\u7f6e\u53d1\u73b0PG\u5728\u591a\u4e2aOSDs\u4e2d\u7684\u6570\u636e\u4e0d\u4e00\u81f4\u540e\uff0c\u662f\u5426\u8fdb\u884c\u4fee\u590d\uff0c\u63a8\u8350\u8bbe\u7f6e\u4e3atrue\uff0c\u5373\u8868\u793a\u81ea\u52a8\u8fdb\u884c\u4fee\u590d\u3002CEPH\u7cfb\u7edf\u9ed8\u8ba4\u503c\u4e3afalse\u3002\n\uff084\uff09osd_max_scrubs\uff1a\u8fdb\u884cScrub\u65f6\uff0c\u8bbe\u7f6e\u5355\u4e2aOSD\u80fd\u540c\u65f6\u8fdb\u884cScrub\u64cd\u4f5c\u6570\uff0c\u63a8\u8350\u8bbe\u7f6e\u4e3a10\u3002CEPH\u7cfb\u7edf\u9ed8\u8ba4\u503c\u662f1\uff0c\u4f1a\u5bfc\u81f4\u6574\u4e2a\u7cfb\u7edf\u80fd\u540c\u65f6\u8fdb\u884c\u7684Scrub\u64cd\u4f5c\u6570\u5f88\u5c11\u3002\u63d0\u9ad8\u8be5\u503c\u80fd\u660e\u663e\u589e\u52a0Scrub\u7684\u5e76\u884c\u6570\uff0c\u589e\u52a0Scrub\u901f\u5ea6\u3002\u5426\u5219\uff0c\u53ef\u80fd\u5bfc\u81f4pgs not deep-scrubbed in time\u7684\u8b66\u62a5\u3002\n\uff085\uff09osd_scrub_min_interval\uff1a\u8bbe\u7f6e\u5355\u4e2aPG\u8fdb\u884cScrub\u7684\u6700\u5c0f\u95f4\u9694\u65f6\u95f4\uff0c\u9ed8\u8ba4\u503c\u4e3a86400\uff0c\u4e00\u5929\u3002\n\uff086\uff09osd_scrub_max_interval\uff1a\u8bbe\u7f6e\u5355\u4e2aPG\u8fdb\u884cScrub\u7684\u6700\u5927\u95f4\u9694\u65f6\u95f4\uff0c\u9ed8\u8ba4\u503c\u4e3a604800\uff0c\u4e00\u5468\u3002\n\uff087\uff09osd_scrub_interval_randomize_ratio\uff1a\u8bbe\u7f6e\u5355\u4e2aPG\u8fdb\u884cScrub\u65f6\uff0c\u95f4\u9694\u4f1a\u989d\u5916\u589e\u52a0\u4e00\u5b9a\u7684\u968f\u673a\u65f6\u95f4\uff0c\u8be5\u65f6\u95f4\u4e3aosd_scrub_min_interval*\u8be5\u9608\u503c\uff0c\u9ed8\u8ba4\u503c\u4e3a0.5\u3002\n\uff088\uff09osd_deep_scrub_randomize_ratio\uff1a\u5728\u8fdb\u884cScrub\u65f6\uff0c\u968f\u673a\u53d8\u66f4\u4e3adeep scrub\u7684\u6982\u7387\uff0c\u63a8\u8350\u8bbe\u7f6e\u4e3a0.05\u3002CEPH\u7cfb\u7edf\u9ed8\u8ba4\u503c\u4e3a0.15\u3002\n\uff089\uff09osd_deep_scrub_interval\uff1a\u8bbe\u7f6e\u8fdb\u884cdeep Scrub\u7684\u65f6\u95f4\u95f4\u9694\uff0c\u63a8\u8350\u8bbe\u7f6e\u4e3a259200\uff0c\u5373\u4e00\u670830\u5929\u3002CEPH\u7cfb\u7edf\u9ed8\u8ba4\u503c\u4e3a604800\uff0c\u5373\u82e5\u6709PG\u6ca1\u80fd\u5728\u4e00\u5468\u5185\u8fdb\u884cdeep scrub\uff0c\u5219\u5bfc\u81f4pgs not deep-scrubbed in time\u7684\u8b66\u62a5\u3002\u6539\u53d8\u8be5\u503c\u5e94\u8be5\u5e76\u4e0d\u80fd\u51cf\u6162\u6216\u52a0\u5febdeep scrub\u7684\u901f\u5ea6\u3002\n<\/pre>\n\n\n\n<p>\u4ee5\u4e0a\u53c2\u6570\u8868\u793a\uff1a\u5bf9\u6bcf\u4e2aPG\u8fdb\u884c\u6e05\u7406\uff0c\u6bcf\u96941~1.5\u5929\u5373\u4f1a\u8fdb\u884c\u4e00\u6b21Scrub\u3002\u6bcf\u6b21\u8fdb\u884cScrub\u65f6\uff0c\u67095%\u7684\u6982\u7387\u53d8\u66f4\u4e3adeep scrub\uff0c\u8fd9\u8868\u793a\u5e73\u574720~30\u5929\u4f1a\u5bf9\u6240\u6709PGs\u8fdb\u884c\u4e00\u8f6edeep Scrub\u3002\u5f53\u4e00\u4e2a\u661f\u671f\u540e\uff0c\u82e5\u6709PG\u672a\u8fdb\u884cscrub\uff0c\u6216\u4e00\u4e2a\u6708\u540e\uff0c\u82e5\u6709PG\u672a\u8fdb\u884cdeep-scrub\uff0c\u5219\u8fdb\u884c\u62a5\u8b66\u3002\u8fdb\u884c\u4f18\u5316\u65b9\u5f0f\uff1a\u901a\u8fc7\u589e\u5927 osd_max_scrubs \u53c2\u6570\u503c\u6765\u52a0\u5febscrub\u901f\u5ea6\uff1b\u901a\u8fc7\u51cf\u5c11 osd_deep_scrub_randomize_ratio \u53c2\u6570\u503c\u6765\u51cf\u5c11deep scrub\u4efb\u52a1\u91cf\uff0c\u51cf\u5c11\u5bf9\u78c1\u76d8\u7684\u8bfb\u53d6\u6d88\u8017\u3002<\/p>\n\n\n\n<h2>2. \u8fdb\u884cPG backfilling\u7684\u53c2\u6570\u548c\u539f\u7406<\/h2>\n\n\n\n<p>\u5f53CEPH\u7cfb\u7edf\u66f4\u6362\u786c\u76d8\u65f6\uff0c\u4f1a\u5c06\u4e00\u4e9bPGs\u8f6c\u79fb\u5230\u65b0\u786c\u76d8\u505a\u7684OSD\u4e0a\uff0c\u8fd9\u65f6\u9700\u8981\u5bf9\u6570\u636e\u8fdb\u884c\u56de\u586b\u3002\u9ed8\u8ba4\u60c5\u51b5\u4e0b\uff0c\u56de\u586b\u901f\u5ea6\u6781\u6162\uff0c\u9700\u8981\u4fee\u6539OSD\u53c2\u6570\u589e\u52a0\u56de\u586b\u901f\u5ea6\u3002<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">\uff0810\uff09osd_max_backfills\uff1a\u8bbe\u7f6eCEPH\u7cfb\u7edf\u4e00\u6b21\u6700\u591a\u5bf9\u6307\u5b9a\u6570\u91cf\u7684PGs\u8fdb\u884c\u56de\u586b\uff0c\u63a8\u8350\u6839\u636e\u9700\u8981\u56de\u586b\u7684PGs\u6570\u91cf\u548cCEPH\u7cfb\u7edf\u7684\u6574\u4f53OSDs\u6570\u91cf\u6765\u8bbe\u7f6e\u3002\u9ed8\u8ba4\u503c\u4e3a1\uff0c\u8fc7\u5c0f\uff0c\u4f1a\u5bfc\u81f4\u56de\u586b\u901f\u5ea6\u6781\u6162\uff0c\u53ef\u80fd\u9700\u8981\u6570\u5929\u624d\u80fd\u8ba9\u6570\u636e\u4fee\u590d\u5b8c\u6bd5\u3002\u54b1\u4eec\u7cfb\u7edf\u6709\u4e00\u6b21\u66f4\u6362\u786c\u76d8\u540e\u9700\u8981\u5bf966\u4e2aPGs\u8fdb\u884c\u56de\u586b\uff0c\u6211\u76f4\u63a5\u8bbe\u7f6e\u8be5\u503c\u4e3a66\u5373\u53ef\u3002\u63a8\u8350\u8be5\u503c * PG\u526f\u672c\u6570 \/ OSDs\u603b\u6570 \u4e0d\u8981\u8d85\u8fc7 3\uff0c\u5373\u8ba9\u6bcf\u4e2a\u786c\u76d8\u540c\u65f6\u7684\u8bfb\u548c\u5199\u5e76\u53d1\u4e0d\u8981\u8fc7\u5927\u3002\n\uff0811\uff09osd_recovery_max_active\uff1a\u8fdb\u884c\u6570\u636e\u6062\u590d\u65f6\uff0c\u8bbe\u7f6e\u6bcf\u4e2aOSD\u7684\u6700\u5927\u8bfb\u5199\u8bf7\u6c42\u6570\uff0c\u63a8\u8350\u8bbe\u7f6e\u4e3a10\u3002CEPH\u7cfb\u7edf\u9ed8\u8ba4\u503c\u4e3a3\u3002\n\uff0812\uff09osd_recovery_max_active_hdd\uff1a\u8fdb\u884c\u6570\u636e\u6062\u590d\u65f6\uff0c\u8bbe\u7f6e\u6bcf\u4e2aOSD\u7684\u673a\u68b0\u786c\u76d8\u6700\u5927\u8bfb\u5199\u8bf7\u6c42\u6570\uff0c\u63a8\u8350\u8bbe\u7f6e\u4e3a10\u3002CEPH\u7cfb\u7edf\u9ed8\u8ba4\u503c\u4e3a3\u3002\n\uff0813\uff09osd_recovery_max_active_ssd\uff1a\u8fdb\u884c\u6570\u636e\u6062\u590d\u65f6\uff0c\u8bbe\u7f6e\u6bcf\u4e2aOSD\u7684\u56fa\u6001\u786c\u76d8\u6700\u5927\u8bfb\u5199\u8bf7\u6c42\u6570\uff0c\u63a8\u8350\u8bbe\u7f6e\u4e3a100\u3002CEPH\u7cfb\u7edf\u9ed8\u8ba4\u503c\u4e3a10\u3002<\/pre>\n\n\n\n<h2>3. \u901a\u8fc7\u547d\u4ee4\u884c\u4fee\u6539OSD\u53c2\u6570<\/h2>\n\n\n\n<p>\u9700\u8981\u5728\u5404\u53f0CEPH\u5b58\u50a8\u670d\u52a1\u5668\u4e0a\u8fd0\u884c\u7a0b\u5e8f\uff0c\u5bf9\u5404\u81ea\u7684OSDs\u8fdb\u884c\u53c2\u6570\u4fee\u6539\u3002\u4e14\u901a\u8fc7\u547d\u4ee4\u884c\u4fee\u6539\u4ec5\u5728\u5f53\u524dCEPH\u7cfb\u7edf\u4e2d\u751f\u6548\uff0c\u91cd\u542f\u670d\u52a1\u5668\u540e\u5931\u6548\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ceph tell osd.0 injectargs --osd_scrub_begin_hour=22 --osd_scrub_end_hour=8 --osd_scrub_auto_repair=true --osd_max_scrubs=10 --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0.5 --osd_deep_scrub_randomize_ratio=0.05 --osd_deep_scrub_interval=259200 --osd_max_backfills=100 --osd_recovery_max_active=10 --osd_recovery_max_active_hdd=10 --osd_recovery_max_active_ssd=100<\/code><\/pre>\n\n\n\n<p>\u67e5\u770b\u4fee\u6539\u540e\u7684\u53c2\u6570\u7ed3\u679c\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ceph daemon osd.0 config show | egrep \"osd_scrub_begin_hour|osd_scrub_end_hour|osd_scrub_auto_repair|osd_max_scrubs|osd_scrub_min_interval|osd_scrub_max_interval|osd_scrub_interval_randomize_ratio|osd_deep_scrub_randomize_ratio|osd_deep_scrub_interval|osd_max_backfills|osd_recovery_max_active|osd_recovery_max_active_hdd|osd_recovery_max_active_ssd\"<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>1. \u8fdb\u884cPG Srub\u7684\u53c2\u6570\u548c\u539f\u7406 \u5bf9PG\u8fdb\u884cScrub\uff0c\u5373\u68c0\u6d4bPG\u5206\u5e03\u5230\u5404O &hellip; <a href=\"http:\/\/www.chenlianfu.com\/?p=3669\">\u7ee7\u7eed\u9605\u8bfb <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"http:\/\/www.chenlianfu.com\/index.php?rest_route=\/wp\/v2\/posts\/3669"}],"collection":[{"href":"http:\/\/www.chenlianfu.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.chenlianfu.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.chenlianfu.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.chenlianfu.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3669"}],"version-history":[{"count":2,"href":"http:\/\/www.chenlianfu.com\/index.php?rest_route=\/wp\/v2\/posts\/3669\/revisions"}],"predecessor-version":[{"id":3672,"href":"http:\/\/www.chenlianfu.com\/index.php?rest_route=\/wp\/v2\/posts\/3669\/revisions\/3672"}],"wp:attachment":[{"href":"http:\/\/www.chenlianfu.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3669"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.chenlianfu.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3669"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.chenlianfu.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3669"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}