Jast blog
Jast-zsh
2023-03-10
目录

协处理器使用

方法 作用
prePut put操作之前执行
postPut put操作之后执行

# 代码

# 安装协处理器

alter 'test_0104_2' , METHOD =>'table_att','coprocessor'=>'/hbasec/a-coprocessor-1.0.1.jar|com.hbase.coprocessor.UserRelatedInformationObserver|1001|arg1=1,arg2=2'

协处理器框架将尝试从协处理器表属性值中读取类信息，读取coprocessor。该值包含由管道（|）字符分隔的四条信息。

  • 文件路径：包含协处理器实现的 jar 文件必须位于所有区域服务器都可以读取它的位置。您可以将文件复制到每个区域服务器上的本地磁盘上，但建议将其存储在 HDFS 中。 HBASE-14548 (opens new window) 允许指定包含 jar 或某些通配符的目录，例如：hdfs：// ： / user / /或 hdfs：// ： / user / /*.jar。请注意，如果指定了目录，则会添加目录中的所有 jar 文件（.jar）。它不搜索子目录中的文件。如果要指定目录，请不要使用通配符。此增强功能也适用于通过 JAVA API 的用法。
  • 类名：协处理器的完整类名。
  • 优先级：整数。该框架将使用优先级确定在同一个钩子上注册的所有已配置观察者的执行顺序。此字段可以留空。在这种情况下，框架将分配默认优先级值。
  • 参数（可选）：此字段传递给协处理器实现。这是可选的。（后面介绍如何在协处理器中调用）

# 安装样例

//创建表
create 'test_table','fn'
//disable
disable 'test_table'
//安装协处理器
alter 'test_table' , METHOD =>'table_att','coprocessor'=>'/hbasec/hbase-coprocessor-245-0.0.1.jar|io.gitee.jastee.hbase.coprocessor.ObserverTemplate|1001'
//enable
enable 'test_table'
//查看表详情确认是否安装
desc 'test_table'
1
2
3
4
5
6
7
8
9
10

create 'test_table','fn' disable 'test_table' alter 'test_table' , METHOD =>'table_att','coprocessor'=>'/hbasec/hbase-coprocessor-245-0.0.1.jar|io.gitee.jastee.hbase.coprocessor.ObserverTemplate|1001' enable 'test_table' desc 'test_table'

# 卸载

create 'test_0104_4','fn'
disable 'invoicing_information'
alter 'invoicing_information', METHOD => 'table_att_unset', NAME => 'coprocessor$1'
enable 'invoicing_information'
disable 'invoicing_information'
alter 'invoicing_information' , METHOD =>'table_att','coprocessor'=>'/hbasec/a-coprocessor-1.1.0.jar|com.hbase.coprocessor.UserRelatedInformationObserver|1001|tableName=invoicing_information'
enable 'invoicing_information'
desc 'invoicing_information'
1
2
3
4
5
6
7
8
9

put 'invoicing_information','r1','fn:one_id','df2'

put 'test_0104_4','r1','fn:test_data','dsfasdfasdf'

put 'test_0104_4','r2','fn:one_id','df2'

put 'test_0104_4','r2','fn:test_data','nullhaha'

put 'test_0104_4','r3','fn:one_id','df2'

put 'test_0104_4','r3','fn:test_data','dsfasdfasdf'

put 'test_0104_4','r4','fn:one_id','df2'

put 'test_0104_4','r4','fn:test_data','nullhaha'

put 'test_0104_4','r5','fn:one_id','df'

put 'test_0104_4','r5','fn:test_data','dsfasdfasdf'

put 'test_0104_4','r6','fn:one_id','df'

put 'test_0104_4','r6','fn:test_data','nullhaha'

put 'test_0104_4','r7','fn:one_id','df3'

put 'test_0104_4','r7','fn:test_data','dsfasdfasdf'

put 'test_0104_4','r8','fn:one_id','df3'

put 'test_0104_4','r8','fn:test_data','nullhaha'

get 'test_0104_4','df3'

get 'test_0104_4','df'

# 可能存在异常

# java.lang.LinkageError: loader constraint violation: loader

2023-01-04 15:14:31,367 ERROR [RS_OPEN_REGION-regionserver/10.16.0.78:6002-3] coprocessor.CoprocessorHost: The coprocessor com.hbase.coprocessor.UserRelatedInformationObserver threw java.lang.LinkageError: loader constraint violation: loader (instance of org/apache/hadoop/hbase/util/CoprocessorClassLoader) previously initiated loading for a different type with name "org/apache/hadoop/conf/Configuration"
java.lang.LinkageError: loader constraint violation: loader (instance of org/apache/hadoop/hbase/util/CoprocessorClassLoader) previously initiated loading for a different type with name "org/apache/hadoop/conf/Configuration"
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        at org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:317)
        at org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:289)
        at com.hbase.coprocessor.UserRelatedInformationObserver.start(UserRelatedInformationObserver.java:48)
        at org.apache.hadoop.hbase.coprocessor.BaseEnvironment.startup(BaseEnvironment.java:72)
        at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:283)
        at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:247)
        at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:198)
        at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:392)
        at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:280)
        at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:914)
        at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:783)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:7957)
        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegionFromTableDir(HRegion.java:8189)
        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:8166)
        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:8124)
        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:8075)
        at org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:149)
        at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2023-01-04 15:14:31,370 ERROR [RS_OPEN_REGION-regionserver/10.16.0.78:6002-3] regionserver.HRegionServer: ***** ABORTING region server 10.16.0.78,6002,1672816432319: The coprocessor com.hbase.coprocessor.UserRelatedInformationObserver threw java.lang.LinkageError: loader constraint violation: loader (instance of org/apache/hadoop/hbase/util/CoprocessorClassLoader) previously initiated loading for a different type with name "org/apache/hadoop/conf/Configuration" *****
java.lang.LinkageError: loader constraint violation: loader (instance of org/apache/hadoop/hbase/util/CoprocessorClassLoader) previously initiated loading for a different type with name "org/apache/hadoop/conf/Configuration"
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        at org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:317)
        at org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:289)
        at com.hbase.coprocessor.UserRelatedInformationObserver.start(UserRelatedInformationObserver.java:48)
        at org.apache.hadoop.hbase.coprocessor.BaseEnvironment.startup(BaseEnvironment.java:72)
        at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:283)
        at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:247)
        at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:198)
        at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:392)
        at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:280)
        at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:914)
        at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:783)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:7957)
        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegionFromTableDir(HRegion.java:8189)
        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:8166)
        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:8124)
        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:8075)
        at org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:149)
        at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2023-01-04 15:14:31,370 ERROR [RS_OPEN_REGION-regionserver/10.16.0.78:6002-3] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [org.apache.hadoop.hbase.security.token.TokenProvider]
2023-01-04 15:14:31,412 WARN  [RS_OPEN_REGION-regionserver/10.16.0.78:6002-3] handler.AssignRegionHandler: Fatal error occurred while opening region test_0104,,1672816335168.987df954f559ab2f0d031e55aea16fca., aborting...
org.apache.hadoop.hbase.regionserver.RegionServerAbortedException: Server 10.16.0.78,6002,1672816432319 aborting
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1569)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2340)
        at org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:163)
        at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2023-01-04 15:14:31,725 ERROR [main] regionserver.HRegionServerCommandLine: Region server exiting
java.lang.RuntimeException: HRegionServer Aborted
        at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:67)
        at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:152)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3189)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

原因：

自己引用的jar包与集群自带的冲突

解决方法：

集群自带的包不打进我们程序中

<build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.1.0</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <transformers>
                                <transformer                         implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.hbase.coprocessor.UserRelatedInformationObserver</mainClass>
                                </transformer>
                            </transformers>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

参考

https://blog.51cto.com/fengzhen/5373832

https://www.bbsmax.com/A/kvJ3wQLpJg/

https://blog.csdn.net/xdkb159/article/details/108347027

https://static.kancloud.cn/apachecn/hbase-doc-zh/1948530

上次更新: 2023/05/11, 16:05:55
HbaseJMX监控
Hbase基于Snapshot数据迁移

← Hbase基于Snapshot数据迁移→

最近更新
01
Linux可视化监控
02-26
02
Maven私服搭建
02-26
03
当ElasticSearch时间字段设置多个格式到底是用的哪个？
01-19
更多文章>
Theme by Vdoing | Copyright © 2016-2025 Jast-zsh | MIT License