kettle 从hive导数据到mysql报错，请帮忙看一下是什么原因

Jane_x

用Kettle从hive到mysql导数据时（数据量有些大，大概6000万+），导到700多万时报错，利用kettle的错误信息如下：
2018/04/04 12:53:29 - 表输出.0 - linenr 7400000
2018/04/04 12:54:07 - 表输入.0 - linenr 7450000
2018/04/04 12:54:09 - 表输出.0 - linenr 7450000
org.Pentaho.di.core.exception.KettleDatabaseException:
Couldn't get row from result set
java.io.IOException: java.lang.RuntimeException: java.util.concurrent.RejectedExecutionException: Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@41d71f56 rejected from java.util.concurrent.ThreadPoolExecutor@45ab8304[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3]
        at org.pentaho.di.core.database.Database.getRow(Database.java:2374)
        at org.pentaho.di.core.database.Database.getRow(Database.java:2344)
        at org.pentaho.di.trans.steps.tableinput.TableInput.processRow(TableInput.java:145)
        at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hive.service.cli.HiveSQLException: java.io.IOException: java.lang.RuntimeException: java.util.concurrent.RejectedExecutionException: Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@41d71f56 rejected from java.util.concurrent.ThreadPoolExecutor@45ab8304[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3]
        at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:256)
        at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:242)
        at org.apache.hive.jdbc.HiveQueryResultSet.next(HiveQueryResultSet.java:365)
        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.pentaho.hadoop.shim.common.DriverProxyInvocationChain$ResultSetInvocationHandler.invoke(DriverProxyInvocationChain.java:756)
        at com.sun.proxy.$Proxy41.next(Unknown Source)
        at org.pentaho.di.core.database.Database.getRow(Database.java:2362)
        ... 4 more

2018/04/04 12:54:32 - 表输入.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Unexpected error
2018/04/04 12:54:32 - 表输入.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseException:
2018/04/04 12:54:32 - 表输入.0 - Couldn't get row from result set
2018/04/04 12:54:32 - 表输入.0 - java.io.IOException: java.lang.RuntimeException: java.util.concurrent.RejectedExecutionException: Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@41d71f56 rejected from java.util.concurrent.ThreadPoolExecutor@45ab8304[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 3]
2018/04/04 12:54:32 - 表输入.0 -
2018/04/04 12:54:32 - 表输入.0 -         at org.pentaho.di.core.database.Database.getRow(Database.java:2374)
2018/04/04 12:54:32 - 表输入.0 -         at org.pentaho.di.core.database.Database.getRow(Database.java:2344)
2018/04/04 12:54:32 - 表输入.0 -         at org.pentaho.di.trans.steps.tableinput.TableInput.processRow(TableInput.java:145)
2018/04/04 12:54:32 - 表输入.0 -         at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2018/04/04 12:54:32 - 表输入.0 -         at java.lang.Thread.run(Thread.java:748)

HitachiVantara

你的数据量太大了 hive jdbc接口不适合
您hive底层怎么存储的？text csv parquet orc？
在pentaho企业版中支持直接读写hadoop文件直接输出到mysql是不是会更快？

www.hitachivantara.com
help.penthao.com