{"id":1004,"date":"2018-05-11T20:41:52","date_gmt":"2018-05-11T12:41:52","guid":{"rendered":"https:\/\/www.xiaobo.li\/?p=1004"},"modified":"2018-05-11T22:56:21","modified_gmt":"2018-05-11T14:56:21","slug":"ceph-pg-pgs","status":"publish","type":"post","link":"https:\/\/www.xiaobo.li\/notes\/archives\/1004","title":{"rendered":"CEPH pg\/pgs"},"content":{"rendered":"
\n

PG\u5168\u79f0\u662fplacement groups\uff0c\u5b83\u662fceph\u7684\u903b\u8f91\u5b58\u50a8\u5355\u5143\u3002\u5728\u6570\u636e\u5b58\u50a8\u5230cesh\u65f6\uff0c\u5148\u6253\u6563\u6210\u4e00\u7cfb\u5217\u5bf9\u8c61\uff0c\u518d\u7ed3\u5408\u57fa\u4e8e\u5bf9\u8c61\u540d\u7684\u54c8\u5e0c\u64cd\u4f5c\u3001\u590d\u5236\u7ea7\u522b\u3001PG\u6570\u91cf\uff0c\u4ea7\u751f\u76ee\u6807PG\u53f7\u3002\u6839\u636e\u590d\u5236\u7ea7\u522b\u7684\u4e0d\u540c\uff0c\u6bcf\u4e2aPG\u5728\u4e0d\u540c\u7684OSD\u4e0a\u8fdb\u884c\u590d\u5236\u548c\u5206\u53d1\u3002\u53ef\u4ee5\u628aPG\u60f3\u8c61\u6210\u5b58\u50a8\u4e86\u591a\u4e2a\u5bf9\u8c61\u7684\u903b\u8f91\u5bb9\u5668\uff0c\u8fd9\u4e2a\u5bb9\u5668\u6620\u5c04\u5230\u591a\u4e2a\u5177\u4f53\u7684OSD\u3002PG\u5b58\u5728\u7684\u610f\u4e49\u662f\u63d0\u9ad8ceph\u5b58\u50a8\u7cfb\u7edf\u7684\u6027\u80fd\u548c\u6269\u5c55\u6027\u3002<\/p>\n

\"\"<\/p>\n

\u5982\u679c\u6ca1\u6709PG\uff0c\u5c31\u96be\u4ee5\u7ba1\u7406\u548c\u8ddf\u8e2a\u6570\u4ee5\u4ebf\u8ba1\u7684\u5bf9\u8c61\uff0c\u5b83\u4eec\u5206\u5e03\u5728\u6570\u767e\u4e2aOSD\u4e0a\u3002\u5bf9ceph\u6765\u8bf4\uff0c\u7ba1\u7406PG\u6bd4\u76f4\u63a5\u7ba1\u7406\u6bcf\u4e2a\u5bf9\u8c61\u8981\u7b80\u5355\u5f97\u591a\u3002\u6bcf\u4e2aPG\u9700\u8981\u6d88\u8017\u4e00\u5b9a\u7684\u7cfb\u7edf\u8d44\u6e90\u5305\u62ecCPU\u3001\u5185\u5b58\u7b49\u3002\u96c6\u7fa4\u7684PG\u6570\u91cf\u5e94\u8be5\u88ab\u7cbe\u786e\u8ba1\u7b97\u5f97\u51fa\u3002\u901a\u5e38\u6765\u8bf4\uff0c\u589e\u52a0PG\u7684\u6570\u91cf\u53ef\u4ee5\u51cf\u5c11OSD\u7684\u8d1f\u8f7d\uff0c\u4f46\u662f\u8fd9\u4e2a\u589e\u52a0\u5e94\u8be5\u6709\u8ba1\u5212\u8fdb\u884c\u3002\u4e00\u4e2a\u63a8\u8350\u914d\u7f6e\u662f\u6bcfOSD\u5bf9\u5e9450-100\u4e2aPG\u3002\u5982\u679c\u6570\u636e\u89c4\u6a21\u589e\u5927\uff0c\u5728\u96c6\u7fa4\u6269\u5bb9\u7684\u540c\u65f6PG\u6570\u91cf\u4e5f\u9700\u8981\u8c03\u6574\u3002CRUSH\u4f1a\u7ba1\u7406PG\u7684\u91cd\u65b0\u5206\u914d\u3002<\/p>\n

\u6bcf\u4e2apool\u5e94\u8be5\u5206\u914d\u591a\u5c11\u4e2aPG\uff0c\u4e0eOSD\u7684\u6570\u91cf\u3001\u590d\u5236\u4efd\u6570\u3001pool\u6570\u91cf\u6709\u5173\uff0c\u6709\u4e2a\u8ba1\u7b97\u516c\u5f0f\u5728\uff1a<\/p>\n

http:\/\/ceph.com\/pgcalc\/<\/a><\/p>\n

\u300alearning ceph\u300b\u8fd9\u672c\u4e66\u91cc\u7684\u8ba1\u7b97\u516c\u5f0f\u4e5f\u5dee\u4e0d\u591a\uff1a<\/p>\n

Total PGs = ((Total_number_of_OSD * 100) \/ max_replication_count) \/ pool_count<\/span><\/span><\/pre>\n
\u7ed3\u7b97\u7684\u7ed3\u679c\u5f80\u4e0a\u53d6\u9760\u8fd12\u7684N\u6b21\u65b9\u7684\u503c\u3002\u6bd4\u5982\u603b\u5171OSD\u6570\u91cf\u662f160\uff0c\u590d\u5236\u4efd\u65703\uff0cpool\u6570\u91cf\u4e5f\u662f3\uff0c\u90a3\u4e48\u6309\u4e0a\u8ff0\u516c\u5f0f\u8ba1\u7b97\u51fa\u7684\u7ed3\u679c\u662f1777.7\u3002\u53d6\u8ddf\u5b83\u63a5\u8fd1\u76842\u7684N\u6b21\u65b9\u662f2048\uff0c\u90a3\u4e48\u6bcf\u4e2apool\u5206\u914d\u7684PG\u6570\u91cf\u5c31\u662f2048\u3002<\/p>\n
\u5728\u66f4\u6539pool\u7684PG\u6570\u91cf\u65f6\uff0c\u9700\u540c\u65f6\u66f4\u6539PGP\u7684\u6570\u91cf\u3002PGP\u662f\u4e3a\u4e86\u7ba1\u7406placement\u800c\u5b58\u5728\u7684\u4e13\u95e8\u7684PG\uff0c\u5b83\u548cPG\u7684\u6570\u91cf\u5e94\u8be5\u4fdd\u6301\u4e00\u81f4\u3002\u5982\u679c\u4f60\u589e\u52a0pool\u7684pg_num\uff0c\u5c31\u9700\u8981\u540c\u65f6\u589e\u52a0pgp_num\uff0c\u4fdd\u6301\u5b83\u4eec\u5927\u5c0f\u4e00\u81f4\uff0c\u8fd9\u6837\u96c6\u7fa4\u624d\u80fd\u6b63\u5e38rebalancing\u3002\u4e0b\u9762\u4ecb\u7ecd\u5982\u4f55\u4fee\u6539pg_num\u548cpgp_num\u3002<\/p>\n
\uff081\uff09\u68c0\u67e5rbd\u8fd9\u4e2apool\u91cc\u5df2\u5b58\u5728\u7684PG\u548cPGP\u6570\u91cf\uff1a<\/p>\n
$ ceph osd pool get rbd pg_num\r\npg_num: 128\r\n$ ceph osd pool get rbd pgp_num\r\npgp_num: 128<\/span><\/span><\/pre>\n
\uff082\uff09\u68c0\u67e5pool\u7684\u590d\u5236size\uff0c\u6267\u884c\u5982\u4e0b\u547d\u4ee4\uff1a<\/p>\n
$ ceph osd dump |grep size|grep rbd\r\npool 2 'rbd' replicated size 3 min_size 2 crush_ruleset 0 object_hash \r\nrjenkins pg_num 128 pgp_num 128 last_change 45 flags hashpspool stripe_width 0<\/span><\/span><\/span><\/pre>\n
\uff083\uff09\u4f7f\u7528\u4e0a\u8ff0\u516c\u5f0f\uff0c\u6839\u636eOSD\u6570\u91cf\u3001\u590d\u5236size\u3001pool\u7684\u6570\u91cf\uff0c\u8ba1\u7b97\u51fa\u65b0\u7684PG\u6570\u91cf\uff0c\u5047\u8bbe\u662f256.<\/p>\n
\uff084\uff09\u53d8\u66f4rbd\u7684pg_num\u548cpgp_num\u4e3a256\uff1a<\/p>\n
$ ceph osd pool set rbd pg_num 256\r\n$ ceph osd pool set rbd pgp_num 256<\/span><\/span><\/pre>\n
\uff085\uff09\u5982\u679c\u6709\u5176\u4ed6pool\uff0c\u540c\u6b65\u8c03\u6574\u5b83\u4eec\u7684pg_num\u548cpgp_num\uff0c\u4ee5\u4f7f\u8d1f\u8f7d\u66f4\u52a0\u5747\u8861\u3002<\/p>\n<\/div>\n
##############################################################<\/p>\n
Ceph\u5e38\u7528\u547d\u4ee4\u603b\u7ed3<\/h1>\n
\n
<\/div>\n<\/div>\n
\n
1. \u521b\u5efa\u81ea\u5b9a\u4e49pool<\/p>\n
ceph<\/span> osd pool create pg_num pgp_num\r\n<\/pre>\n
\u5176\u4e2dpgp_num\u4e3apg_num\u7684\u6709\u6548\u5f52\u7f6e\u7ec4\u4e2a\u6570\uff0c\u662f\u4e00\u4e2a\u53ef\u9009\u53c2\u6570\u3002pg_num\u5e94\u8be5\u8db3\u591f\u5927\uff0c\u4e0d\u8981\u62d8\u6ce5\u4e8e\u5b98\u65b9\u6587\u6863\u7684\u8ba1\u7b97\u65b9\u6cd5\uff0c\u6839\u636e\u5b9e\u9645\u60c5\u51b5\u9009\u62e9256\u3001512\u30011024\u30012048\u30014096\u3002<\/p>\n
2. \u8bbe\u7f6epool\u7684\u526f\u672c\u6570\u3001\u6700\u5c0f\u526f\u672c\u6570\u3001\u6700\u5927\u526f\u672c\u6570<\/p>\n
ceph osd pool set<\/span> <poolname> size 2<\/span>\r\nceph osd pool set<\/span> <poolname> min_size 1<\/span>\r\nceph osd pool set<\/span> <poolname> max_size 10<\/span><\/pre>\n
\u8d44\u6e90\u6240\u9650\uff0c\u5982\u679c\u4e0d\u5e0c\u671b\u4fdd\u5b583\u526f\u672c\uff0c\u53ef\u4ee5\u7528\u8be5\u547d\u4ee4\u5bf9\u7279\u5b9a\u7684pool\u66f4\u6539\u526f\u672c\u5b58\u653e\u6570\u3002<\/p>\n
\u5229\u7528get\u53ef\u4ee5\u83b7\u5f97\u7279\u5b9apool\u7684\u526f\u672c\u6570\u3002<\/p>\n
ceph osd pool get<\/span> <poolname> size\r\n<\/pre>\n
3. \u589e\u52a0osd<\/p>\n
\u53ef\u4ee5\u5229\u7528ceph-deploy\u589e\u52a0osd\uff1a<\/p>\n
ceph-deploy osd prepare monosd1:\/mnt\/ceph osd2:\/mnt\/ceph\r\nceph-deploy osd activate monosd1:\/mnt\/ceph osd2:\/mnt\/ceph\r\n\r\n#<\/span>\u76f8\u5f53\u4e8e\uff1a<\/span>\r\nceph-deploy osd create monosd1:\/mnt\/ceph osd2:\/mnt\/ceph\r\n\r\n#<\/span>\u8fd8\u6709\u4e00\u79cd\u65b9\u6cd5\uff0c\u5728\u5b89\u88c5osd\u65f6\u540c\u65f6\u6307\u5b9a\u5bf9\u5e94\u7684journal\u7684\u5b89\u88c5\u8def\u5f84<\/span>\r\nceph-deploy osd create osd1:\/cephmp1:\/dev\/sdf1 \/cephmp2:\/dev\/sdf2\r\n<\/pre>\n
\u4e5f\u53ef\u4ee5\u624b\u52a8\u589e\u52a0\uff1a<\/p>\n
## Prepare disk first, create partition and format it<\/span>\r\n<insert parted oneliner>\r\nmkfs.xfs -f \/dev\/sdd\r\nmkdir<\/span> \/cephmp1\r\nmount \/dev\/sdd \/cephmp1\r\ncd \/cephmp1\r\n\r\nceph-osd -i 12<\/span> --mkfs --mkkey\r\nceph auth add osd.12<\/span> osd 'allow *'<\/span> mon 'allow rwx'<\/span> -i \/cephmp1\/keyring\r\n\r\n#change the crushmap<\/span>\r\nceph osd getcrushmap -o map<\/span>\r\ncrushtool -d map<\/span> -o map.txt\r\nvim map.txt\r\ncrushtool -c map.txt -o map<\/span>\r\nceph osd setcrushmap -i map<\/span>\r\n\r\n## Start it<\/span>\r\n\/etc\/init.d\/ceph start osd.12<\/span><\/pre>\n
4. \u5220\u9664osd<\/p>\n
\u5148\u5c06\u6b64osd\u505c\u6b62\u5de5\u4f5c\uff1a<\/p>\n
#<\/span># Mark it out<\/span><\/span>\r\nceph osd out 5\r\n\r\n#<\/span># Wait for data migration to complete (ceph -w), then stop it<\/span><\/span>\r\nservice ceph -a stop osd.5\r\n\r\n#<\/span># Now it is marked out and down<\/span><\/span><\/pre>\n
\u518d\u5bf9\u5176\u8fdb\u884c\u5220\u9664\u64cd\u4f5c\uff1a<\/p>\n
#<\/span># If deleting from active stack, be sure to follow the above to mark it out and down<\/span><\/span>\r\nceph osd crush remove osd.5\r\n\r\n#<\/span># Remove auth for disk<\/span><\/span>\r\nceph auth del osd.5\r\n\r\n#<\/span># Remove disk<\/span><\/span>\r\nceph osd rm 5\r\n\r\n#<\/span># Remove from ceph.conf and copy new conf to all hosts<\/span><\/span><\/pre>\n
5. \u67e5\u770bosd\u603b\u4f53\u60c5\u51b5\u3001osd\u7684\u8be6\u7ec6\u4fe1\u606f\u3001crush\u7684\u8be6\u7ec6\u4fe1\u606f<\/p>\n
ceph osd tree\r\nceph osd dump<\/span> --format<\/span>=json-pretty\r\nceph osd crush dump<\/span> --format<\/span>=json-pretty\r\n<\/pre>\n
6. \u83b7\u5f97\u5e76\u4fee\u6539CRUSH maps<\/p>\n
#<\/span># save current crushmap in binary<\/span><\/span>\r\nceph osd getcrushmap -o crushmap.bin\r\n\r\n#<\/span># Convert to txt<\/span><\/span>\r\ncrushtool -d crushmap.bin -o crushmap.txt\r\n\r\n#<\/span># Edit it and re-convert to binary<\/span><\/span>\r\ncrushtool -c crushmap.txt -o crushmap.bin.new\r\n\r\n#<\/span># Inject into running system<\/span><\/span>\r\nceph osd setcrushmap -i crushmap.bin.new\r\n\r\n#<\/span># If you've added a new ruleset and want to use that for a pool, do something like:<\/span><\/span>\r\nceph osd pool default crush rule = 4\r\n\r\n#<\/span>\u4e5f\u53ef\u4ee5\u8fd9\u6837\u8bbe\u7f6e\u4e00\u4e2apool\u7684rule<\/span>\r\ncpeh osd pool set testpool crush_ruleset <ruleset_id>\r\n<\/pre>\n
-o=output; -d=decompile; -c=compile; -i=input<\/p>\n
\u8bb0\u4f4f\u8fd9\u4e9b\u7f29\u5199\uff0c\u4e0a\u9762\u7684\u547d\u4ee4\u5c31\u5f88\u5bb9\u6613\u7406\u89e3\u4e86\u3002<\/p>\n
7. \u589e\u52a0\/\u5220\u9664journal<\/p>\n
\u4e3a\u4e86\u63d0\u9ad8\u6027\u80fd\uff0c\u901a\u5e38\u5c06ceph\u7684journal\u7f6e\u4e8e\u5355\u72ec\u7684\u78c1\u76d8\u6216\u5206\u533a\u4e2d\uff1a<\/p>\n
\u5148\u5229\u7528\u4ee5\u4e0b\u547d\u4ee4\u8bbe\u7f6eceph\u96c6\u7fa4\u4e3anodown\uff1a<\/p>\n
    \n
  • ceph osd set nodown<\/li>\n<\/ul>\n
    # Relevant ceph.conf options<\/span>\r\n -- existing setup --\r\n[osd]\r\n    osd data = \/srv\/ceph\/osd$id<\/span>\r\n    osd journal = \/srv\/ceph\/osd$id<\/span>\/journal\r\n    osd journal size = 512\r\n\r\n# stop the OSD:<\/span>\r\n\/etc\/init.d\/ceph osd.0 stop\r\n\/etc\/init.d\/ceph osd.1 stop\r\n\/etc\/init.d\/ceph osd.2 stop\r\n\r\n# Flush the journal:<\/span>\r\nceph-osd -i 0 --flush-journal\r\nceph-osd -i 1 --flush-journal\r\nceph-osd -i 2 --flush-journal\r\n\r\n# Now update ceph.conf - this is very important or you'll just recreate \r\njournal on the same disk again<\/span>\r\n -- change to [filebased journal] --\r\n[osd]\r\n    osd data = \/srv\/ceph\/osd$id<\/span>\r\n    osd journal = \/srv\/ceph\/journal\/osd$id<\/span>\/journal\r\n    osd journal size = 10000\r\n\r\n -- change to [partitionbased journal (journal\r\n in<\/span> this case<\/span> would be on \/dev\/sda2)] --\r\n[osd]\r\n    osd data = \/srv\/ceph\/osd$id<\/span>\r\n    osd journal = \/dev\/sda2\r\n    osd journal size = 0\r\n\r\n# Create new journal on each disk<\/span>\r\nceph-osd -i 0 --mkjournal\r\nceph-osd -i 1 --mkjournal\r\nceph-osd -i 2 --mkjournal\r\n\r\n# Done, now start all OSD again<\/span>\r\n\/etc\/init.d\/ceph osd.0 start\r\n\/etc\/init.d\/ceph osd.1 start\r\n\/etc\/init.d\/ceph osd.2 start\r\n<\/pre>\n
    \u8bb0\u5f97\u5c06nodown\u8bbe\u7f6e\u56de\u6765\uff1a<\/p>\n
      \n
    • ceph osd unset nodown<\/li>\n<\/ul>\n
      8. ceph cache pool<\/p>\n
      \u7ecf\u521d\u6b65\u6d4b\u8bd5\uff0cceph\u7684cache pool\u6027\u80fd\u5e76\u4e0d\u597d\uff0c\u6709\u65f6\u751a\u81f3\u4f4e\u4e8e\u65e0cache pool\u65f6\u7684\u6027\u80fd\u3002\u53ef\u4ee5\u5229\u7528flashcache\u7b49\u66ff\u4ee3\u65b9\u6848\u6765\u4f18\u5316ceph\u7684cache\u3002<\/p>\n
      ceph osd tier add satapool ssdpool\r\nceph osd tier cache<\/span>-mode<\/span> ssdpool writeback\r\nceph osd pool set<\/span> ssdpool hit_set_type bloom\r\nceph osd pool set<\/span> ssdpool hit_set_count 1<\/span>\r\n\r\n## In<\/span> this example 80<\/span>-85<\/span>% of<\/span> the cache<\/span> pool is<\/span> equal to<\/span> 280<\/span>GB\r\nceph osd pool set<\/span> ssdpool target_max_bytes $((280<\/span>*1024<\/span>*1024<\/span>*1024<\/span>))\r\nceph osd tier<\/span> set<\/span>-overlay satapool ssdpool\r\nceph osd pool set<\/span> ssdpool hit_set_period 300<\/span>\r\nceph osd pool set<\/span> ssdpool cache_min_flush_age 300<\/span>   # 10<\/span> minutes\r\nceph osd pool set<\/span> ssdpool cache_min_evict_age 1800<\/span>   # 30<\/span> minutes\r\nceph osd pool set<\/span> ssdpool cache_target_dirty_ratio .4<\/span>\r\nceph osd pool set<\/span> ssdpool cache_target_full_ratio .8<\/span><\/pre>\n
      9. \u67e5\u770b\u8fd0\u884c\u65f6\u914d\u7f6e<\/p>\n
      ceph --admin-daemon \/var<\/span>\/run\/ceph\/ceph-osd.0<\/span>.asok config show\r\n<\/pre>\n
      10. \u67e5\u770b\u76d1\u63a7\u96c6\u7fa4\u72b6\u6001<\/p>\n
      ceph health\r\ncehp health detail\r\nceph status\r\nceph -s<\/span>\r\n\r\n#\u53ef\u4ee5\u52a0\u4e0a--fortmat=json-pretty<\/span>\r\nceph osd stat<\/span>\r\nceph osd dump<\/span>\r\nceph osd tree\r\n\r\nceph mon stat<\/span>\r\nceph quorum_status\r\nceph mon dump<\/span>\r\n\r\nceph mds stat<\/span>\r\nceph mds dump<\/span><\/pre>\n
      11. \u67e5\u770b\u6240\u6709\u7684pool<\/p>\n
      ceph<\/span> osd lspools\r\nrados lspools\r\n<\/pre>\n
      12. \u67e5\u770bkvm\u548cqemu\u662f\u5426\u652f\u6301rbd<\/p>\n
      qemu-system<\/span>-x86_64 -drive format<\/span>=?\r\nqemu-img -h | grep<\/span> rbd\r\n<\/pre>\n
      13, \u67e5\u770b\u7279\u5b9a\u7684pool\u53ca\u5176\u4e2d\u7684\u6587\u4ef6<\/p>\n
      rbd<\/span> ls testpool\r\nrbd create testpool\/test.img -s 1024<\/span> --image-format=2<\/span>\r\nrbd info<\/span> testpool\/test.img\r\nrbd rm testpool\/test.img\r\n\r\n#\u7edf\u8ba1\u5757\u6570<\/span>\r\nrados -p testpool ls | grep ^rb.0.11a1<\/span> | wc -l\r\n<\/pre>\n
      #\u5bfc\u5165\u5e76\u67e5\u770b\u6587\u4ef6<\/span>\r\nrados<\/span> makepool testpool\r\nrados put -p testpool logo.png logo.png\r\nceph osd map testpool logo.png\r\n\r\nrbd import logo.png testpool\/logo.png\r\nrbd info<\/span> testpool\/logo.png\r\n<\/pre>\n
      14. \u6302\u8f7d\/\u5378\u8f7d\u521b\u5efa\u7684\u5757\u8bbe\u5907<\/p>\n
      ceph osd pool create<\/span> testpool 256<\/span> 256<\/span>\r\nrbd create<\/span> testpool\/test.img -s 1024<\/span> --image-format=2<\/span>\r\nrbd map<\/span> testpool\/test.img\r\nrbd showmapped\r\nmkfs.xfs \/dev\/rbd0\r\nrbd unmap \/dev\/rbd0\r\n<\/pre>\n
      15. \u521b\u5efa\u5feb\u7167<\/p>\n
      #<\/span>\u521b\u5efa<\/span>\r\nrbd snap create testpool\/test.img@test.img-snap1\r\n#<\/span>\u67e5\u770b<\/span>\r\nrbd snap ls testpool\/test.img\r\n#<\/span>\u56de\u6eda<\/span>\r\nrbd snap rollback testpool\/test.img@test.img-snap1\r\n#<\/span>\u5220\u9664<\/span>\r\nrbd snap rm testpool\/test.img@test.img-snap1\r\n#<\/span>\u6e05\u9664\u6240\u6709\u5feb\u7167<\/span>\r\nrbd snap purge testpool\/test.img\r\n<\/pre>\n
      16. \u8ba1\u7b97\u5408\u7406\u7684pg\u6570<\/p>\n
      \u5b98\u65b9\u5efa\u8bae\u6bcfOSD50-100\u4e2apg\u3002total pgs=osds*100\/\u526f\u672c\u6570\uff0c\u4f8b\u59826osd\u30012\u526f\u672c\u7684\u73af\u5883\uff0cpgs\u4e3a6*100\/2=300<\/p>\n
      pg\u6570\u53ea\u80fd\u589e\u52a0\uff0c\u65e0\u6cd5\u51cf\u5c11\uff1b\u589e\u52a0pg_num\u540e\u5fc5\u987b\u540c\u65f6\u589e\u51cfpgp_num<\/p>\n
      17. \u5bf9pool\u7684\u64cd\u4f5c<\/p>\n
      ceph osd pool create<\/span> testpool 256<\/span> 256<\/span>\r\nceph osd pool delete<\/span> testpool testpool --yes-i-really-really-mean-it<\/span>\r\nceph osd pool rename<\/span> testpool anothertestpool\r\nceph osd pool mksnap testpool testpool-snap\r\n<\/pre>\n
      18. \u91cd\u65b0\u5b89\u88c5\u524d\u7684\u683c\u5f0f\u5316<\/p>\n
      ceph-deploy purge<\/span> osd0 osd1\r\nceph-deploy purgedata osd0 osd1\r\nceph-deploy forgetkeys\r\n\r\nceph-deploy disk zap --fs-type xfs osd0:\/dev\/sdb1<\/span><\/pre>\n
      19. \u4fee\u6539osd journal\u7684\u5b58\u50a8\u8def\u5f84<\/p>\n
      #noout\u53c2\u6570\u4f1a\u963b\u6b62osd\u88ab\u6807\u8bb0\u4e3aout\uff0c\u4f7f\u5176\u6743\u91cd\u4e3a0<\/span>\r\nceph osd set<\/span> noout\r\nservice ceph stop osd.1\r\nceph-osd -i 1 --flush-journal\r\nmount \/dev\/sdc \/journal\r\nceph-osd -i 1 --mkjournal \/journal\r\nservice ceph start osd.1\r\nceph osd unset<\/span> noout\r\n<\/pre>\n
      20. xfs\u6302\u8f7d\u53c2\u6570<\/p>\n
      mkfs.xfs -n size=64k \/dev\/sdb1\r\n\r\n#<\/span>\/etc\/fstab\u6302\u8f7d\u53c2\u6570<\/span>\r\nrw,noexec,nodev,noatime,nodiratime,nobarrier\r\n<\/pre>\n
      21. \u8ba4\u8bc1\u914d\u7f6e<\/p>\n
      [global<\/span>]\r\nauth cluser required = none\r\nauth service required = none\r\nauth client required = none\r\n\r\n#0.56\u4e4b\u524d<\/span>\r\nauth supported = none\r\n<\/pre>\n
      22. pg_num\u4e0d\u591f\u7528\uff0c\u8fdb\u884c\u8fc1\u79fb\u548c\u91cd\u547d\u540d<\/p>\n
      ceph osd pool create<\/span> new<\/span>-pool pg_num\r\nrados cppool old<\/span>-pool new<\/span>-pool\r\nceph osd pool delete<\/span> old<\/span>-pool\r\nceph osd pool rename<\/span> new<\/span>-pool old<\/span>-pool\r\n\r\n#\u6216\u8005\u76f4\u63a5\u589e\u52a0pool\u7684pg_num\r\n<\/pre>\n
      23. \u63a8\u9001config\u6587\u4ef6<\/p>\n
      ceph-deploy --overwrite-conf config push<\/span> mon1 mon2 mon3\r\n<\/pre>\n
      24. \u5728\u7ebf\u4fee\u6539config\u53c2\u6570<\/p>\n
      ceph<\/span> tell osd.*<\/span> injectargs '--mon_clock_drift_allowde 1'<\/span><\/pre>\n
      \u4f7f\u7528\u6b64\u547d\u4ee4\u9700\u8981\u533a\u5206\u914d\u7f6e\u7684\u53c2\u6570\u5c5e\u4e8emon\u3001mds\u8fd8\u662fosd\u3002<\/p>\n<\/div>\n
      \n
      Ceph\u4e09\u8282\u70b9\uff083 mon+9 osd\uff09\u96c6\u7fa4\u90e8\u7f72<\/h1>\n
      2017-7-14<\/h1>\n
      \u4e00\u3001\u57fa\u7840\u73af\u5883\u51c6\u5907
      \n3\u53f0\u673a\u5668\uff0c\u6bcf\u53f0\u673a\u5668\u6807\u914d\uff1a1G\u5185\u5b58\u30012\u5757\u7f51\u5361\u30013\u575720G\u7684SATA\u88f8\u76d8
      \n\"$\"\u7b26\u53f7\u8868\u793a\u4e09\u4e2a\u8282\u70b9\u90fd\u8fdb\u884c\u540c\u6837\u914d\u7f6e
      \n$ cat \/etc\/hosts
      \nceph-node1 10.20.0.101
      \nceph-node2 10.20.0.102
      \nceph-node3 10.20.0.103
      \n$ yum install epel-release<\/p>\n
      \u4e8c\u3001\u5b89\u88c5ceph-deploy
      \n[root@ceph-node1 ~]# ssh-keygen \/\/\u5728node1\u8282\u70b9\u914d\u7f6e\u514dSSH\u5bc6\u94a5\u767b\u5f55\u5230\u5176\u4ed6\u8282\u70b9
      \n[root@ceph-node1 ~]# ssh-copy-id ceph-node2
      \n[root@ceph-node1 ~]# ssh-copy-id ceph-node3
      \n[root@ceph-node1 ~]# yum install ceph-deploy -y<\/p>\n
      \u4e09\u4e2a\u8282\u70b9\u5b89\u88c5: yum install ceph<\/p>\n
      [root@ceph01 yum.repos.d]# cat ceph.repo
      \n[ceph]
      \nname=ceph
      \nbaseurl=http:\/\/mirrors.aliyun.com\/ceph\/rpm-jewel\/el7\/x86_64\/
      \ngpgcheck=0
      \n[ceph-noarch]
      \nname=cephnoarch
      \nbaseurl=http:\/\/mirrors.aliyun.com\/ceph\/rpm-jewel\/el7\/noarch\/
      \ngpgcheck=0
      \n[root@ceph01 yum.repos.d]#<\/p>\n
      [root@ceph-node1 ~]# ceph-deploy new ceph-node1 ceph-node2 ceph-node3 \/\/new\u547d\u4ee4\u4f1a\u751f\u6210\u4e00\u4e2aceph-node1\u96c6\u7fa4\uff0c\u5e76\u4e14\u4f1a\u5728\u5f53\u524d\u76ee\u5f55\u751f\u6210\u914d\u7f6e\u6587\u4ef6\u548c\u5bc6\u94a5\u6587\u4ef6<\/p>\n
      +++++++++++++++++++++++++++++++++++++++
      \n[root@ceph-node1 ~]# ceph -v
      \nceph version 0.94.10 (b1e0532418e4631af01acbc0cedd426f1905f4af)
      \n[root@ceph-node1 ~]# ceph-deploy mon create-initial \/\/\u521b\u5efa\u7b2c\u4e00\u4e2amonitor
      \n[root@ceph-node1 ~]# ceph status \/\/\u6b64\u65f6\u96c6\u7fa4\u5904\u4e8eerror\u72b6\u6001\u662f\u6b63\u5e38\u7684
      \ncluster ea54af9f-f286-40b2-933d-9e98e7595f1a
      \nhealth HEALTH_ERR
      \n[root@ceph-node1 ~]# systemctl start ceph
      \n[root@ceph-node1 ~]# systemctl enable ceph<\/p>\n
       <\/p>\n
      \u4e09\u3001\u521b\u5efa\u5bf9\u8c61\u5b58\u50a8\u8bbe\u5907OSD\uff0c\u5e76\u52a0\u5165\u5230ceph\u96c6\u7fa4
      \n[root@ceph-node1 ~]# ceph-deploy disk list ceph-node1 \/\/\u5217\u51faceph-node1\u5df2\u6709\u7684\u78c1\u76d8\uff0c\u5f88\u5947\u602a\u6ca1\u6709\u5217\u51fasdb\u3001sdc\u3001sdd\uff0c\u4f46\u662f\u786e\u5b9e\u5b58\u5728\u7684
      \n\/\/\u4e0b\u9762\u7684zap\u547d\u4ee4\u614e\u7528\uff0c\u4f1a\u9500\u6bc1\u78c1\u76d8\u4e2d\u5df2\u7ecf\u5b58\u5728\u7684\u5206\u533a\u8868\u548c\u6570\u636e\u3002ceph-node1\u662f\u4e3b\u673a\u540d\uff0c\u540c\u6837\u53ef\u4ee5\u662fceph-node2
      \n[root@ceph-node1 ~]# ceph-deploy disk zap ceph-node1:sdb ceph-node1:sdc ceph-node1:sdd
      \n[root@ceph-node1 ~]# ceph-deploy osd create ceph-node1:sdb ceph-node1:sdc ceph-node1:sdd \/\/\u64e6\u9664\u78c1\u76d8\u539f\u6709\u6570\u636e\uff0c\u5e76\u521b\u5efa\u65b0\u7684\u6587\u4ef6\u7cfb\u7edf\uff0c\u9ed8\u8ba4\u662fXFS\uff0c\u7136\u540e\u5c06\u78c1\u76d8\u7684\u7b2c\u4e00\u4e2a\u5206\u533a\u4f5c\u4e3a\u6570\u636e\u5206\u533a\uff0c\u7b2c\u4e8c\u4e2a\u5206\u533a\u4f5c\u4e3a\u65e5\u5fd7\u5206\u533a\u3002\u52a0\u5165\u5230OSD\u4e2d\u3002
      \n[root@ceph-node1 ~]# ceph status \/\/\u53ef\u4ee5\u770b\u5230\u96c6\u7fa4\u4f9d\u65e7\u6ca1\u6709\u5904\u4e8e\u5065\u5eb7\u72b6\u6001\u3002\u6211\u4eec\u9700\u8981\u518d\u6dfb\u52a0\u4e00\u4e9b\u8282\u70b9\u5230ceph\u96c6\u7fa4\u4e2d\uff0c\u4ee5\u4fbf\u5b83\u80fd\u591f\u5f62\u6210\u5206\u5e03\u5f0f\u7684\u3001\u5197\u4f59\u7684\u5bf9\u8c61\u5b58\u50a8\uff0c\u8fd9\u6837\u96c6\u7fa4\u72b6\u6001\u624d\u4e3a\u5065\u5eb7\u3002
      \ncluster ea54af9f-f286-40b2-933d-9e98e7595f1a
      \nhealth HEALTH_WARN
      \n64 pgs stuck inactive
      \n64 pgs stuck unclean
      \nmonmap e1: 1 mons at {ceph-node1=10.20.0.101:6789\/0}
      \nelection epoch 2, quorum 0 ceph-node1
      \nosdmap e6: 3 osds: 0 up, 0 in
      \npgmap v7: 64 pgs, 1 pools, 0 bytes data, 0 objects
      \n0 kB used, 0 kB \/ 0 kB avail
      \n64 creating<\/p>\n
       <\/p>\n
      \u56db\u3001\u7eb5\u5411\u6269\u5c55\u591a\u8282\u70b9Ceph\u96c6\u7fa4\uff0c\u6dfb\u52a0Monitor\u548cOSD
      \n\u6ce8\u610f\uff1aCeph\u5b58\u50a8\u96c6\u7fa4\u6700\u5c11\u9700\u8981\u4e00\u4e2aMonitor\u5904\u4e8e\u8fd0\u884c\u72b6\u6001\uff0c\u8981\u63d0\u4f9b\u53ef\u7528\u6027\u7684\u8bdd\uff0c\u5219\u9700\u8981\u5947\u6570\u4e2amonitor\uff0c\u6bd4\u59823\u4e2a\u62165\u4e2a\uff0c\u4ee5\u5f62\u6210\u4ef2\u88c1\uff08quorum\uff09\u3002
      \n\uff081\uff09\u5728Ceph-node2\u548cCeph-node3\u90e8\u7f72monitor\uff0c\u4f46\u662f\u662f\u5728ceph-node1\u6267\u884c\u547d\u4ee4\uff01
      \n[root@ceph-node1 ~]# ceph-deploy mon add ceph-node2
      \n[root@ceph-node1 ~]# ceph-deploy mon add ceph-node3
      \n++++++++++++++++++++++++++
      \n\u62a5\u9519\uff1a[root@ceph-node1 ~]# ceph-deploy mon create ceph-node2
      \n[ceph-node3][WARNIN] Executing \/sbin\/chkconfig ceph on
      \n[ceph-node3][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
      \n[ceph-node3][WARNIN] monitor: mon.ceph-node3, might not be running yet
      \n[ceph-node3][INFO ] Running command: ceph --cluster=ceph --admin-daemon \/var\/run\/ceph\/ceph-mon.ceph-node3.asok mon_status
      \n[ceph-node2][WARNIN] neither `public_addr` nor `public_network` keys are defined for monitors
      \n\u89e3\u51b3\uff1a\u2460\u901a\u8fc7\u5728CentOS 7\u4e0achkconfig\uff0c\u6000\u7591\u8282\u70b91\u5e76\u6ca1\u6709\u8fdc\u7a0b\u542f\u52a8\u8282\u70b92\u7684ceph\u670d\u52a1\uff0c\u5982\u679c\u6211\u5728node2\u624b\u52a8\u542f\u52a8\u7684\u8bdd\uff0c\u5e94\u8be5\u5c31\u53ef\u4ee5\u4e86
      \n[root@ceph-node2 ~]# systemctl status ceph
      \n\u25cf ceph.service - LSB: Start Ceph distributed file system daemons at boot time
      \nLoaded: loaded (\/etc\/rc.d\/init.d\/ceph)
      \nActive: inactive (dead)
      \n\u7ed3\u679cenable\u540e\u8fd8\u662f\u5931\u8d25\u4e86
      \n\u2461\u6c83\u65e5\uff0c\u539f\u6765\u662f\u4e66\u4e0a\u5199\u9519\u4e86\uff0c\u5728\u5df2\u7ecf\u6dfb\u52a0\u4e86\u76d1\u63a7\u8282\u70b9\u540e\uff0c\u540e\u7eed\u6dfb\u52a0\u76d1\u63a7\u8282\u70b9\u5e94\u8be5\u662fmon add\uff0c\u771f\u7684\u662f\u9189\u4e86\uff01<\/p>\n
      ++++++++++++++++++++++++++++++++
      \n[root@ceph-node1 ~]# ceph status
      \nmonmap e3: 3 mons at {ceph-node1=10.20.0.101:6789\/0,ceph-node2=10.20.0.102:6789\/0,ceph-node3=10.20.0.103:6789\/0}
      \nelection epoch 8, quorum 0,1,2 ceph-node1,ceph-node2,ceph-node3<\/p>\n
      \uff082\uff09\u6dfb\u52a0\u66f4\u591a\u7684OSD\u8282\u70b9\uff0c\u4f9d\u7136\u5728ceph-node1\u6267\u884c\u547d\u4ee4\u5373\u53ef\u3002
      \n[root@ceph-node1 ~]# ceph-deploy disk list ceph-node2 ceph-node3
      \n\/\/\u786e\u4fdd\u78c1\u76d8\u53f7\u4e0d\u8981\u51fa\u9519\uff0c\u5426\u5219\u7684\u8bdd\uff0c\u5bb9\u6613\u628a\u7cfb\u7edf\u76d8\u90fd\u7ed9\u683c\u5f0f\u5316\u4e86\uff01
      \n[root@ceph-node1 ~]# ceph-deploy disk zap ceph-node2:sdb ceph-node2:sdc ceph-node2:sdd
      \n[root@ceph-node1 ~]# ceph-deploy disk zap ceph-node3:sdb ceph-node3:sdc ceph-node3:sdd
      \n\/\/\u7ecf\u8fc7\u5b9e\u8df5\uff0c\u4e0b\u9762\u7684\u8fd9\u6761\u547d\u4ee4\uff0cosd create\u6700\u597d\u5206\u4e24\u6b65\uff0cprepare\u548cactivate\uff0c\u7ec8\u4e8e\u4e3a\u4ec0\u4e48\u4e0d\u6e05\u695a\u3002
      \n[root@ceph-node1 ~]# ceph-deploy osd create ceph-node2:sdb ceph-node2:sdc ceph-node2:sdd
      \n[root@ceph-node1 ~]# ceph-deploy osd create ceph-node3:sdb ceph-node3:sdc ceph-node3:sdd<\/p>\n
      ++++++++++++++++++++++++++++++++++++++++++++
      \n\u62a5\u9519\uff1a
      \n[ceph-node3][WARNIN] ceph-disk: Error: Command '['\/usr\/sbin\/sgdisk', '--new=2:0:5120M', '--change-name=2:ceph journal', '--partition-guid=2:fa28bc46-55de-464a-8151-9c2b51f9c00d', '--typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106', '--mbrtogpt', '--', '\/dev\/sdd']' returned non-zero exit status 4
      \n[ceph-node3][ERROR ] RuntimeError: command returned non-zero exit status: 1
      \n[ceph_deploy.osd][ERROR ] Failed to execute command: ceph-disk -v prepare --fs-type xfs --cluster ceph -- \/dev\/sdd
      \n[ceph_deploy][ERROR ] GenericError: Failed to create 3 OSDs
      \n\u672a\u89e3\u51b3\uff1a\u539f\u6765\u6572\u5165osd create\u547d\u4ee4\u4e0d\u5c0f\u5fc3\u628anode2\u5199\u6210node3\u4e86\uff0c\u54ce\u6211\u5c3c\u739b\uff0c\u540e\u9762\u8d8a\u6765\u8d8a\u96be\u529e\u4e86
      \n+++++++++++++++++++++++++++++<\/p>\n
      [root@ceph-node1 ~]# ceph osd tree
      \nID WEIGHT TYPE NAME UP\/DOWN REWEIGHT PRIMARY-AFFINITY
      \n-1 0.08995 root default
      \n-2 0.02998 host ceph-node1
      \n0 0.00999 osd.0 up 1.00000 1.00000
      \n1 0.00999 osd.1 up 1.00000 1.00000
      \n2 0.00999 osd.2 up 1.00000 1.00000
      \n-3 0.02998 host ceph-node2
      \n3 0.00999 osd.3 down 0 1.00000
      \n4 0.00999 osd.4 down 0 1.00000
      \n8 0.00999 osd.8 down 0 1.00000
      \n-4 0.02998 host ceph-node3
      \n5 0.00999 osd.5 down 0 1.00000
      \n6 0.00999 osd.6 down 0 1.00000
      \n7 0.00999 osd.7 down 0 1.00000<\/p>\n
      \u67096\u4e2aOSD\u90fd\u5904\u4e8edown\u72b6\u6001\uff0cceph-deploy osd activate \u4f9d\u7136\u5931\u8d25\uff0c\u6839\u636e\u4e4b\u524d\u7684\u62a5\u544a\uff0cosd create\u7684\u65f6\u5019\u5c31\u662f\u5931\u8d25\u7684\u3002
      \n\u672a\u89e3\u51b3:\u7531\u4e8e\u521a\u90e8\u7f72ceph\u96c6\u7fa4\uff0c\u8fd8\u6ca1\u6709\u6570\u636e\uff0c\u53ef\u4ee5\u628aOSD\u7ed9\u6e05\u7a7a\u3002
      \n\u53c2\u8003\u6587\u6863\uff1ahttp:\/\/www.cnblogs.com\/zhangzhengyan\/p\/5839897.html
      \n[root@ceph-node1 ~]# ceph-deploy disk zap ceph-node2:sdb ceph-node2:sdc ceph-node2:sdd
      \n[root@ceph-node1 ~]# ceph-deploy disk zap ceph-node3:sdb ceph-node3:sdc ceph-node3:sdd<\/p>\n
      \uff081\uff09\u4ececeph osd tree\u79fb\u8d70crush map\u7684osd.4\uff0c\u8fd8\u6709\u5176\u4ed6osd\u53f7
      \n[root@ceph-node1 ~]# ceph osd crush remove osd.3
      \n[root@ceph-node1 ~]# ceph osd crush remove osd.4
      \n[root@ceph-node1 ~]# ceph osd crush remove osd.8
      \n[root@ceph-node1 ~]# ceph osd crush remove osd.5
      \n[root@ceph-node1 ~]# ceph osd crush remove osd.6
      \n[root@ceph-node1 ~]# ceph osd crush remove osd.7<\/p>\n
      \uff082\uff09[root@ceph-node1 ~]# ceph osd rm 3
      \n[root@ceph-node1 ~]# ceph osd rm 4
      \n[root@ceph-node1 ~]# ceph osd rm 5
      \n[root@ceph-node1 ~]# ceph osd rm 6
      \n[root@ceph-node1 ~]# ceph osd rm 7
      \n[root@ceph-node1 ~]# ceph osd rm 8<\/p>\n
      [root@ceph-node1 ~]# ceph osd tree \/\/\u7ec8\u4e8e\u6e05\u7406\u5e72\u51c0\u4e86
      \nID WEIGHT TYPE NAME UP\/DOWN REWEIGHT PRIMARY-AFFINITY
      \n-1 0.02998 root default
      \n-2 0.02998 host ceph-node1
      \n0 0.00999 osd.0 up 1.00000 1.00000
      \n1 0.00999 osd.1 up 1.00000 1.00000
      \n2 0.00999 osd.2 up 1.00000 1.00000
      \n-3 0 host ceph-node2
      \n-4 0 host ceph-node3
      \n\u53ef\u4ee5\u767b\u5f55\u5230node2\u548cnode3\uff0csdb\/sdc\/sdd\u90fd\u88ab\u5e72\u6389\u4e86\uff0c\u9664\u4e86\u8fd8\u5269GPT\u683c\u5f0f\uff0c\u73b0\u5728\u91cd\u65b0ceph-deploy osd create
      \n\u6211\u5c3c\u739b\u8fd8\u662f\u62a5\u9519\u5440\uff0c[ceph_deploy.osd][ERROR ] Failed to execute command: ceph-disk -v prepare --fs-type xfs --cluster ceph -- \/dev\/sdd<\/p>\n
      ++++++++++++++++++++++++++++++++++
      \n\u62a5\u9519\uff1a\uff081\uff09node1\u8fdc\u7a0b\u6fc0\u6d3bnode2\u7684osd\u51fa\u9519\u3002prepare\u548cactivate\u80fd\u591f\u53d6\u4ee3osd create\u7684\u6b65\u9aa4
      \n[root@ceph-node1 ~]# ceph-deploy osd prepare ceph-node2:sdb ceph-node2:sdc ceph-node2:sdd
      \n[root@ceph-node1 ~]# ceph-deploy osd activate ceph-node2:sdb ceph-node2:sdc ceph-node2:sdd<\/p>\n
      [ceph-node2][WARNIN] ceph-disk: Cannot discover filesystem type: device \/dev\/sdb: Line is truncated:
      \n[ceph-node2][ERROR ] RuntimeError: command returned non-zero exit status: 1
      \n[ceph_deploy][ERROR ] RuntimeError: Failed to execute command: ceph-disk -v activate --mark-init sysvinit --mount \/dev\/sdb
      \n\u89e3\u51b3\uff1a\u683c\u5f0f\u5206\u533a\u6743\u9650\u95ee\u9898\uff0c\u5728\u62a5\u9519\u7684\u8282\u70b9\u6267\u884cceph-disk activate-all\u5373\u53ef\u3002<\/p>\n
      \uff082\uff09\u660e\u660e\u662fnode2\u7684sdb\u76d8\uff0c\u4f46\u662fceph osd tree\u5374\u53d1\u73b0\u6267\u884c\u7684\u662fnode3\u7684sdb\u76d8\uff0c\u5f53\u7136\u4f1a\u62a5\u9519\u4e86
      \nStarting Ceph osd.4 on ceph-node2...
      \nRunning as unit ceph-osd.4.1500013086.674713414.service.
      \nError EINVAL: entity osd.3 exists but key does not match
      \n[root@ceph-node1 ~]# ceph osd tree
      \n3 0 osd.3 down 0 1.00000<\/p>\n
      \u89e3\u51b3\uff1a[root@ceph-node1 ~]# ceph auth del osd.3
      \n[root@ceph-node1 ~]# ceph osd rm 3
      \n\u5728node2\u4e0alsblk\u53d1\u73b0sdb\u4e0d\u6b63\u5e38\uff0c\u6ca1\u6709\u6302\u8f7dosd\uff0c\u90a3\u4e48\u4e8e\u662f
      \n[root@ceph-node1 ~]# ceph-deploy disk zap ceph-node2:sdb
      \n[root@ceph-node1 ~]# ceph-deploy osd prepare ceph-node2:sdb
      \n[root@ceph-node1 ~]# ceph osd tree \/\/\u81f3\u5c11osd\u8dd1\u5230node2\u4e0a\u9762\uff0c\u800c\u4e0d\u662fnode3\uff0c\u8fd8\u597d\u8fd8\u597d\u3002
      \n-3 0.02998 host ceph-node2
      \n3 0.00999 osd.3 down 0 1.00000
      \n[root@ceph-node1 ~]# ceph-deploy osd activate ceph-node2:sdb \/\/\u80af\u5b9a\u5931\u8d25\uff0c\u6309\u7167\u4e0a\u9762\u7684\u7ecf\u9a8c\uff0c\u5fc5\u987b\u5728Node2\u4e0a\u5355\u72ec\u6fc0\u6d3b
      \n[root@ceph-node2 ~]# ceph-disk activate-all<\/p>\n
      [root@ceph-node1 ~]# ceph osd tree
      \nID WEIGHT TYPE NAME UP\/DOWN REWEIGHT PRIMARY-AFFINITY
      \n-1 0.08995 root default
      \n-2 0.02998 host ceph-node1
      \n0 0.00999 osd.0 up 1.00000 1.00000
      \n1 0.00999 osd.1 up 1.00000 1.00000
      \n2 0.00999 osd.2 up 1.00000 1.00000
      \n-3 0.02998 host ceph-node2
      \n4 0.00999 osd.4 up 1.00000 1.00000
      \n5 0.00999 osd.5 up 1.00000 1.00000
      \n3 0.00999 osd.3 up 1.00000 1.00000
      \n-4 0.02998 host ceph-node3
      \n6 0.00999 osd.6 up 1.00000 1.00000
      \n7 0.00999 osd.7 up 1.00000 1.00000
      \n8 0.00999 osd.8 up 1.00000 1.00000
      \n\u54ce\uff0c\u5367\u69fd\uff0c\u7ec8\u4e8e\u89e3\u51b3\u4e86\uff0c\u4e4b\u524d\u53ea\u662f\u4e00\u4e2a\u5c0f\u5c0f\u7684\u76d8\u7b26\u5199\u9519\u4e86\uff0c\u5c31\u5bb3\u5f97\u6211\u641e\u8fd9\u4e48\u4e45\u554a\uff0c\u7ec6\u5fc3\u70b9\uff01<\/p>\n
      =========================================
      \n\u62d3\u5c55\uff1a
      \n[root@ceph-node1 ~]# lsblk \/\/ OSD up\u7684\u5206\u533a\u90fd\u6302\u8f7d\u5230\/var\/lib\/ceph\/osd\u76ee\u5f55\u4e0b
      \nNAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
      \nsda 8:0 0 40G 0 disk
      \n\u251c\u2500sda1 8:1 0 500M 0 part \/boot
      \n\u2514\u2500sda2 8:2 0 39.5G 0 part
      \n\u251c\u2500centos-root 253:0 0 38.5G 0 lvm \/
      \n\u2514\u2500centos-swap 253:1 0 1G 0 lvm [SWAP]
      \nsdb 8:16 0 20G 0 disk
      \n\u251c\u2500sdb1 8:17 0 15G 0 part \/var\/lib\/ceph\/osd\/ceph-0
      \n\u2514\u2500sdb2 8:18 0 5G 0 part
      \nsdc 8:32 0 20G 0 disk
      \n\u251c\u2500sdc1 8:33 0 15G 0 part \/var\/lib\/ceph\/osd\/ceph-1
      \n\u2514\u2500sdc2 8:34 0 5G 0 part
      \nsdd 8:48 0 20G 0 disk
      \n\u251c\u2500sdd1 8:49 0 15G 0 part \/var\/lib\/ceph\/osd\/ceph-2
      \n\u2514\u2500sdd2 8:50 0 5G 0 part
      \nsr0 11:0 1 1024M 0 rom<\/p>\n
       <\/p>\n
       <\/p>\n
       <\/p>\n
       <\/p>\n
       <\/p>\n
       <\/p>\n","protected":false},"excerpt":{"rendered":"
      PG\u5168\u79f0\u662fplacement groups\uff0c\u5b83\u662fceph\u7684\u903b\u8f91\u5b58\u50a8\u5355\u5143\u3002\u5728\u6570\u636e\u5b58 … \u7ee7\u7eed\u9605\u8bfb →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[292],"tags":[264],"class_list":["post-1004","post","type-post","status-publish","format-standard","hentry","category-storage","tag-ceph"],"_links":{"self":[{"href":"https:\/\/www.xiaobo.li\/notes\/wp-json\/wp\/v2\/posts\/1004","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.xiaobo.li\/notes\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.xiaobo.li\/notes\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.xiaobo.li\/notes\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.xiaobo.li\/notes\/wp-json\/wp\/v2\/comments?post=1004"}],"version-history":[{"count":0,"href":"https:\/\/www.xiaobo.li\/notes\/wp-json\/wp\/v2\/posts\/1004\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.xiaobo.li\/notes\/wp-json\/wp\/v2\/media?parent=1004"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.xiaobo.li\/notes\/wp-json\/wp\/v2\/categories?post=1004"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.xiaobo.li\/notes\/wp-json\/wp\/v2\/tags?post=1004"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}