Shulou(Shulou.com)06/01 Report--

Kylin reports an error when executing the query statement as follows:

Error while executing SQL "select t. Failing current query... (check region server log), d.day_of_week LIMIT 50000": The coprocessor thread stopped itself due to scan timeout or scan threshold (check region server log), failing current query...

View regionserver Log

2017-03-20 11 5842625beb89 10 INFO [Query dc7017bb-fefc-4177-a2c9-5842625beb89-109] endpoint.CubeVisitService: Scanned 9999001 rows from HBase.

2017-03-20 11 5842625beb89 10 INFO [Query dc7017bb-fefc-4177-a2c9-5842625beb89-109] endpoint.CubeVisitService: The cube visit did not finish normally because scan num exceeds threshold

Org.apache.kylin.gridtable.GTScanExceedThresholdException: Exceed scan threshold at 10000001

At org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService$1.hasNext (CubeVisitService.java:258)

At org.apache.kylin.storage.hbase.cube.v2.HBaseReadonlyStore$1 $1.hasNext (HBaseReadonlyStore.java:111)

At org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube (CubeVisitService.java:290)

At org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod (CubeVisitProtos.java:4117)

At org.apache.hadoop.hbase.regionserver.HRegion.execService (HRegion.java:7797)

At org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion (RSRpcServices.java:1982)

At org.apache.hadoop.hbase.regionserver.RSRpcServices.execService (RSRpcServices.java:1964)

At org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod (ClientProtos.java:33652)

At org.apache.hadoop.hbase.ipc.RpcServer.call (RpcServer.java:2170)

At org.apache.hadoop.hbase.ipc.CallRunner.run (CallRunner.java:109)

At org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run (RpcExecutor.java:185)

At org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run (RpcExecutor.java:165)

2017-03-20 11 ipc.RpcServer: (responseTooSlow): {"call": "ExecService (org.apache.hadoop.hbase.protobuf.generated.ClientProtos$CoprocessorServiceRequest)", "starttimems": 1489979376495, "responsesize": 359, "method": "ExecService", "processingtimems": 28964, "client": "10.10.16.102 class" 58720 "," queuetimems ": 1," class ":" HRegionServer "}

Analysis and solution:

There is a configuration item in the ${KYLIN_HOME} / conf/kylin.property configuration file

Kylin.query.scan.threshold=10000000

This configuration item limits the number of rows in the Hbase scan. When the scan exceeds the kylin.query.scan.threshold row, it still cannot meet the query requirements, then the kylin cancels the query on the hbase side.

At present, there are two ways to solve the problem.

1. Increase the value of kylin.query.scan.holded, which will correspondingly increase the hbase pressure. If Hbase can withstand the query pressure, you can increase this value.

2. In this query statement, LIMIT 50000J HBASE cannot scan out 50000 rows after scan the kylin.query.scan.threshold rows. The value of LIMIT can be reduced so that Hbase can meet the query requirements before scan reaches kylin.query.scan.threshold. This problem will not arise.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.