A. 澶ф暟鎹涓甯歌佹暟鎹瀛樺偍鏍煎紡涓庡帇緙╂牸寮
鍦ㄥぇ鏁版嵁鐨勪笘鐣岄噷錛屾暟鎹瀛樺偍鍜屽帇緙╂妧宸ф槸鑷沖叧閲嶈佺殑涓鐜銆傚畠浠涓嶄粎鍏充箮絀洪棿鏁堢巼錛屾洿鐩存帴褰卞搷鐫鏁版嵁澶勭悊鐨勯熷害鍜屾ц兘銆傝╂垜浠娣卞叆鎺㈣ㄥ嚑縐嶅父瑙佺殑鏁版嵁瀛樺偍鏍煎紡浠ュ強鍘嬬緝鎶鏈錛屼互婊¤凍澶氭牱鍖栫殑鏌ヨ㈤渶奼傚拰浼樺寲璧勬簮鍒╃敤銆
棣栧厛錛岃╂垜浠鐪嬬湅鏁版嵁瀛樺偍鐨勪笁縐嶄富瑕佹ā寮忥細
1. 琛屽紡瀛樺偍錛圤LTP錛錛屽傚悓縐鏈ㄨ埇鎸夎屽爢縐錛岄傜敤浜庨戠箒鐨勫啓鍏ユ搷浣滐紝濡傚叧緋誨瀷鏁版嵁搴撱傝
鍒楀紡瀛樺偍錛圤LAP錛鍒欎互鍒椾負涓誨礆紝渚誇簬鏁版嵁鍒嗘瀽錛岀壒鍒閫傚悎璇誨彇瀵嗛泦鍨嬪満鏅錛屾瘮濡傛暟鎹浠撳簱錛圖W錛夊拰鏁版嵁婀栵紙DA錛夈
涓轟簡鍏奸【涓嶅悓鏌ヨ㈡ц兘錛屾垜浠榪樺紩鍏ヤ簡娣峰悎瀛樺偍錛屽閥濡欒瀺鍚堜簡涓よ呯殑浼樺娍錛屼互閫傚簲鍚勭嶆煡璇㈤渶奼傘
- Parquet錛屽垪寮忓瓨鍌ㄧ殑鏄庢槦錛屼笓涓烘暟鎹鍒嗘瀽璁捐★紝閫氳繃鍘嬬緝鍒楁暟鎹錛屾敮鎸佸嶆潅綾誨瀷錛屾槸Impala鍜孒ive鐨勯栭夈
- ORC錛孒ive鐨勪笓灞炲垪瀛樻牸寮忥紝鏀鎸乑LIB鍜孲NAPPY鍘嬬緝錛屽規暟鎹浠撳簱鐨勮誨彇浼樺寲琛ㄧ幇鍑鴻壊銆
- TextFile錛岃櫧鐒朵互琛屽瓨褰㈠紡瀛樺湪錛屼絾鏃犲帇緙╀笖瑙ｆ瀽鎴愭湰楂橈紝涓昏佸簲鐢ㄤ簬鏁版嵁鐨勫垵濮嬪眰錛圤DS錛夈
- SequenceFile錛孒adoop鐨勫熀鐭籌紝閲囩敤浜岃繘鍒跺簭鍒楀寲錛岄敭鍊煎圭粨鏋勶紝涓昏佺敤浜庤剼鏈鍔犺澆錛岄潪鍘嬬緝銆
鍦ㄤ紬澶氬瓨鍌ㄦ牸寮忎腑錛
Hive鐨凷equenceFile鐙鍏風壒鑹詫紝瀹冪畝鍖栦簡MR錛圡apRece錛夌殑map闃舵墊帓搴忥紝鎻愰珮鏁堢巼銆傝
AVRO鍒欐槸涓涓寮哄ぇ鐨勬暟鎹搴忓垪鍖栫郴緇燂紝鏀鎸佷赴瀵岀殑鏁版嵁緇撴瀯錛屾棤闇鐢熸垚浠ｇ爜鍗沖彲涓庡姩鎬佽璦鏃犵紳瀵規帴銆
鑷充簬鏁版嵁鍘嬬緝錛岄夋嫨鑷沖叧閲嶈併傚父瑙侀夐」鍖呮嫭錛
- Deflate錛氬鉤琛″帇緙╂瘮鍜岄熷害錛孒adoop鏀鎸侊紝浣嗕笉鏀鎸佸垎鐗囥
- Snappy錛氶熷害椋炲揩錛孒adoop鍘熺敓鏀鎸侊紝浣嗗帇緙╃巼鐩稿硅緝浣庯紝涓嶆敮鎸佸垎鐗囷紝閫傚悎鐢熶駭鐜澧冦
- GZIP錛氬帇緙╂瘮楂橈紝Hadoop鍏煎癸紝浣嗕笉鏀鎸佸垎鐗囥
- LZO錛氶熷害蹇錛屾敮鎸佸垎鐗囷紝浣嗗帇緙╃巼杈冧綆錛岄渶瑕侀濆栧畨瑁呫
- SNAPPY錛氶熷害涓嶴nappy鐩稿綋錛屽師鐢熸敮鎸侊紝鍘嬬緝鐜囪緝浣庯紝浣嗕笉鏀鎸佸垎鐗囥
- BZIP2錛氬帇緙╃巼鏋侀珮錛屼絾閫熷害杈冩參錛岃嚜甯﹀懡浠ゅ湪Linux涓婂彲鐢錛孒adoop鏀鎸佷絾闈炲師鐢熴
鍦ㄥ疄闄呭簲鐢ㄤ腑錛屾暟鎹鍒嗗眰絳栫暐鏄鍏抽敭銆傚湪ODS灞傦紝涓轟簡鑺傜渷絀洪棿錛屾垜浠鍊懼悜浜庨夋嫨濡俍LIB銆丟ZIP鎴朆ZIP2榪欐牱鐨勯珮鍘嬬緝鐜囩畻娉曘傝屽湪DW灞傚拰DA灞傦紝鐢變簬瀵歸熷害鍜屾晥鐜囪佹眰杈冮珮錛孲nappy鐨勯珮鏁堣В鍘嬬緝鎴愪負浼橀夈
緇間笂鎵榪幫紝閫夋嫨鏁版嵁瀛樺偍鏍煎紡鍜屽帇緙╂妧鏈鏃訛紝鍔″繀鏍規嵁鏁版嵁鐨勭壒鎬у拰搴旂敤鍦烘櫙榪涜岀簿緇嗚瘎浼幫紝浠ュ疄鐜版渶浣崇殑鎬ц兘鍜屽瓨鍌ㄦ晥鐜囥傛暟鎹縐戝﹀跺拰宸ョ▼甯堜滑錛岃繖灝嗘槸浣犱紭鍖栧ぇ鏁版嵁澶勭悊榪囩▼鐨勫疄鐢ㄦ寚鍗椼