你见过不长毛的羊吗

Hbase 启动失败,Please check the config value of 'hbase.procedure.store.wal.use.hsync'

2019.01.08

基本环境

环境: JDK1.8 Hadoop 2.7.1 Zookeeper 3.4.6 Hbase 2.0.0 Apache Phoenix-5.0.0-Hbase-2.0.0

问题描述

从hadoop到hbase的环境搭建一切正常,可直接参照网上教程,这里不累述。

这里记录一下我在配置phoenix连接hbase的时候遇到的问题。 直观的看是HMaster先能启动,过一会又自动挂掉。日志错误信息如下:

ERROR [master/YSL-MASTER:16000] master.HMaster: Failed to become active master
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1043)
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:382)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:530)
	at org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1222)
	at org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1141)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2019)
	at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:553)
	at java.lang.Thread.run(Thread.java:748)

官网说明:http://hbase.apache.org/book.html#trouble.master.startup.hsync。 看了之后还是不知道怎么解决。

自己尝试了很多方法,换hadoop版本、修改hbase配置,替换hbase/lib目录下hadoop相关jar包(先删除原jar包),这里记录一下替换命令:find /usr/local/hadoop-2.7.1/share/hadoop -name "hadoop*jar" | xargs -i cp {} /usr/local/hbase-2.0.0/lib/ 。网上也有说换hbase版本(未尝试)。 网上最多的方法就是: hbase/conf/hbase-site.xml增加配置

<property>
	<name>hbase.unsafe.stream.capability.enforce</name>
	<value>false</value>
	<description>
		Controls whether HBase will check for stream capabilities (hflush/hsync).
	    Disable this if you intend to run on LocalFileSystem, denoted by a rootdir
	    with the 'file://' scheme, but be mindful of the NOTE below.
	    WARNING: Setting this to false blinds you to potential data loss and
	    inconsistent system state in the event of process and/or node failures. If
	    HBase is complaining of an inability to use hsync or hflush it's most
	    likely not a false positive.
    </description>
</property>
该配置的作用是:如果你打算在本地文件系统中跑hbase,请禁掉此项。而我这里是将hbase跑在hdfs文件系统上,所以不能配置此项。

解决方法

删除每个节点hbase/lib目录下的phoenix-××-client.jar包,重启hbase即可。 具体原因我也还没搞清楚,以后再更新。 附上:hadoop-hbase 版本对应关系

最后,在这里记录下hbase/conf/hbase-site.xml中添加的配置:

<property>
	<name>phoenix.schema.isNamespaceMappingEnabled</name>
    	<value>true</value>
    	<description>开启schema与namespace的对应关系</description>
</property>
<property>
	<name>phoenix.schema.mapSystemTablesToNamespace</name>
  	<value>true</value>
  	<description>开启schema与namespace的对应关系</description>
</property>
<property>
    	<name>phoenix.functions.allowUserDefinedFunctions</name>
    	<value>true</value>
    	<description>允许用户自定义函数(UDF)</description>
</property>
<property>
	<name>hbase.regionserver.wal.codec</name>
	<value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value>
   	<description>开启phoenix 二级索引</description>
</property>
<property>
    	<name>hbase.regionserver.executor.openregion.threads</name> 
    	<value>100</value>
    	<description>打开用户表region的线程数量(默认3)</description>
</property>
<property>
    	<name>phoenix.default.column.encoded.bytes.attrib</name>
    	<value>0</value>
    	<description>不对hbase列进行编码存储</description>
</property>