Spring Cloud Hystrix ThreadPool的bug

BUG背景

JDK: 11.0.4 Spring Cloud Finchley.SR3

相关配置:

1#开启hystrix 2feign.hystrix.enabled=true 3#关闭断路器 4hystrix.command.default.circuitBreaker.enabled=false 5#禁用hystrix远程调用超时时间 6hystrix.command.default.execution.timeout.enabled=false 7hystrix.threadpool.default.coreSize=50

Hystrix隔离策略:线程隔离

在Feign调用的时候,会报错:

1Caused by: java.util.concurrent.ExecutionException: Observable onError 2 at rx.internal.operators.BlockingOperatorToFuture$2.getValue(BlockingOperatorToFuture.java:118) ~[rxjava-1.3.8.jar!/:1.3.8] 3 at rx.internal.operators.BlockingOperatorToFuture$2.get(BlockingOperatorToFuture.java:102) ~[rxjava-1.3.8.jar!/:1.3.8] 4 at com.netflix.hystrix.HystrixCommand$4.get(HystrixCommand.java:423) ~[hystrix-core-1.5.18.jar!/:1.5.18] 5 at com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:344) ~[hystrix-core-1.5.18.jar!/:1.5.18] 6 ... 30 more 7Caused by: java.lang.IllegalArgumentException 8 at java.util.concurrent.ThreadPoolExecutor.setCorePoolSize(ThreadPoolExecutor.java:1535) ~[?:?] 9 at com.netflix.hystrix.HystrixThreadPool$HystrixThreadPoolDefault.touchConfig(HystrixThreadPool.java:230) ~[hystrix-core-1.5.18.jar!/:1.5.18] 10 at com.netflix.hystrix.HystrixThreadPool$HystrixThreadPoolDefault.getScheduler(HystrixThreadPool.java:205) ~[hystrix-core-1.5.18.jar!/:1.5.18] 11 at com.netflix.hystrix.AbstractCommand.executeCommandWithSpecifiedIsolation(AbstractCommand.java:710) ~[hystrix-core-1.5.18.jar!/:1.5.18] 12 at com.netflix.hystrix.AbstractCommand.executeCommandAndObserve(AbstractCommand.java:638) ~[hystrix-core-1.5.18.jar!/:1.5.18] 13 at com.netflix.hystrix.AbstractCommand.applyHystrixSemantics(AbstractCommand.java:546) ~[hystrix-core-1.5.18.jar!/:1.5.18] 14 at com.netflix.hystrix.AbstractCommand.access$200(AbstractCommand.java:60) ~[hystrix-core-1.5.18.jar!/:1.5.18] 15 at com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:419) ~[hystrix-core-1.5.18.jar!/:1.5.18] 16 at com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:413) ~[hystrix-core-1.5.18.jar!/:1.5.18] 17 at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:46) ~[rxjava-1.3.8.jar!/:1.3.8] 18 at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35) ~[rxjava-1.3.8.jar!/:1.3.8] 19 at rx.Observable.unsafeSubscribe(Observable.java:10327) ~[rxjava-1.3.8.jar!/:1.3.8] 20 at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48) ~[rxjava-1.3.8.jar!/:1.3.8] 21 at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33) ~[rxjava-1.3.8.jar!/:1.3.8] 22 at rx.Observable.unsafeSubscribe(Observable.java:10327) ~[rxjava-1.3.8.jar!/:1.3.8] 23 at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41) ~[rxjava-1.3.8.jar!/:1.3.8] 24 at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30) ~[rxjava-1.3.8.jar!/:1.3.8] 25 at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48) ~[rxjava-1.3.8.jar!/:1.3.8] 26 at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30) ~[rxjava-1.3.8.jar!/:1.3.8] 27 at rx.Observable.unsafeSubscribe(Observable.java:10327) ~[rxjava-1.3.8.jar!/:1.3.8] 28 at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41) ~[rxjava-1.3.8.jar!/:1.3.8] 29 at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30) ~[rxjava-1.3.8.jar!/:1.3.8] 30 at rx.Observable.unsafeSubscribe(Observable.java:10327) ~[rxjava-1.3.8.jar!/:1.3.8] 31 at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:51) ~[rxjava-1.3.8.jar!/:1.3.8] 32

重启有时候复现,有时候不复现,估计是和类加载还有初始化顺序有关的问题。

问题定位

查看问题源代码, 对于每个微服务调用,初始化Hytrix线程池,动态读取配置:

1private void touchConfig() { 2 final int dynamicCoreSize = properties.coreSize().get(); 3 final int configuredMaximumSize = properties.maximumSize().get(); 4 int dynamicMaximumSize = properties.actualMaximumSize(); 5 final boolean allowSizesToDiverge = properties.getAllowMaximumSizeToDivergeFromCoreSize().get(); 6 boolean maxTooLow = false; 7 8 if (allowSizesToDiverge && configuredMaximumSize < dynamicCoreSize) { 9 //if user sets maximum < core (or defaults get us there), we need to maintain invariant of core <= maximum 10 dynamicMaximumSize = dynamicCoreSize; 11 maxTooLow = true; 12 } 13 14 // In JDK 6, setCorePoolSize and setMaximumPoolSize will execute a lock operation. Avoid them if the pool size is not changed. 15 if (threadPool.getCorePoolSize() != dynamicCoreSize || (allowSizesToDiverge && threadPool.getMaximumPoolSize() != dynamicMaximumSize)) { 16 if (maxTooLow) { 17 logger.error("Hystrix ThreadPool configuration for : " + metrics.getThreadPoolKey().name() + " is trying to set coreSize = " + 18 dynamicCoreSize + " and maximumSize = " + configuredMaximumSize + ". Maximum size will be set to " + 19 dynamicMaximumSize + ", the coreSize value, since it must be equal to or greater than the coreSize value"); 20 } 21 threadPool.setCorePoolSize(dynamicCoreSize); 22 threadPool.setMaximumPoolSize(dynamicMaximumSize); 23 } 24 25 threadPool.setKeepAliveTime(properties.keepAliveTimeMinutes().get(), TimeUnit.MINUTES); 26}

报错是在 threadPool.setCorePoolSize(dynamicCoreSize);,看下为何:

1public void setCorePoolSize(int corePoolSize) { 2 if (corePoolSize < 0 || maximumPoolSize < corePoolSize) 3 throw new IllegalArgumentException(); 4 //忽略其他代码 5}

发现还有maximumPoolSize < corePoolSize的判断,所以,需要先设置MaximumPoolSize,之后设置CorePoolSize,这样可以避免这个问题

相关ISSUES与解决方案

看了下社区,已经有ISSUE了:https://github.com/Netflix/Hystrix/issues/1874 还有对应的PULL REQUEST:https://github.com/Netflix/Hystrix/pull/1877

但是目前,还没有合并,只好自己改了,项目中增加同名同路径替换类,修改:

1//jdk9以上的版本,设置coreSize会验证coreSize必须小于maxSize,所以先改变max再设置core 2 threadPool.setMaximumPoolSize(dynamicMaximumSize); 3 threadPool.setCorePoolSize(dynamicCoreSize);
点赞
收藏

评论区

加载中...

相关推荐

MySQL:[Err] 1292 - Incorrect datetime value: ‘0000-00-00 00:00:00‘ for column ‘CREATE_TIME‘ at row 1

文章目录问题用navicat导入数据时,报错:原因这是因为当前的MySQL不支持datetime为0的情况。解决修改sql\mode:sql\mode:SQLMode定义了MySQL应支持的SQL语法、数据校验等,这样可以更容易地在不同的环境中使用MySQL。全局s

Oracle 分组与拼接字符串同时使用

SELECTT.,ROWNUMIDFROM(SELECTT.EMPLID,T.NAME,T.BU,T.REALDEPART,T.FORMATDATE,SUM(T.S0)S0,MAX(UPDATETIME)CREATETIME,LISTAGG(TOCHAR(

MySQL部分从库上面因为大量的临时表tmp_table造成慢查询

背景描述Time:20190124T00:08:14.70572408:00User@Host:@Id:Schema:sentrymetaLast_errno:0Killed:0Query_time:0.315758Lock_

皕杰报表之UUID

​在我们用皕杰报表工具设计填报报表时,如何在新增行里自动增加id呢?能新增整数排序id吗?目前可以在新增行里自动增加id,但只能用uuid函数增加UUID编码,不能新增整数排序id。uuid函数说明:获取一个UUID,可以在填报表中用来创建数据ID语法:uuid()或uuid(sep)参数说明:sep布尔值,生成的uuid中是否包含分隔符'',缺省为

手写Java HashMap源码

HashMap的使用教程HashMap的使用教程HashMap的使用教程HashMap的使用教程HashMap的使用教程22

2020年前端实用代码段,为你的工作保驾护航

有空的时候,自己总结了几个代码段,在开发中也经常使用,谢谢。1、使用解构获取json数据let jsonData  id: 1,status: "OK",data: 'a', 'b';let  id, status, data: number   jsonData;console.log(id, status, number )