当前位置:Java -> Spring RestTemplate 到 WebClient 引起 OutOfMemoryError
Spring Boot是Java企业应用程序中非常流行的框架。一种与内部或外部应用程序集成的常见方法是通过HTTP REST连接,我们正在将从RestTemplate升级到基于Java NIO的WebClient,它可以通过允许并发调用REST服务端点来显著提升应用程序性能。WebClients的好处如下:
尽管性能有所改善,但是在相同数量的并发连接下,WebClient出现了OutOfMemoryError
崩溃。我们将分析WebClient崩溃问题以及如何进行故障排除和修复。
RestTemplate
升级为WebClient
为了充分利用NIO的优势,例如并发和异步处理,我们将从Spring RestTemplate
升级为WebClient,如下所示。
RestTemplate
public void restClientCall(Integer id, String url,String imagePath) {
// Create RestTemplate instance
RestTemplate restTemplate = new RestTemplate();
// Prepare the image file
File imageFile = new File(imagePath);
// Prepare headers
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
// Prepare the request body
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
body.add("file", new org.springframework.core.io.FileSystemResource(imageFile));
// Create the HTTP entity with headers and the multipart body
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body, headers);
System.out.println("Starting to post an image for Id"+id);
// Perform the POST request
ResponseEntity<String> responseEntity = restTemplate.postForEntity(url, requestEntity,
String.class);
// Print the response status code and body
System.out.println("Response Id "+id +":"+ responseEntity.getBody());
System.out.println(" Time: " + LocalTime.now());
}
如下所示的Spring WebClient:
public void webHeavyClientCall(Integer id,String url, String imagePath) {
// Create a WebClient instance
WebClient webClient = WebClient.create();
// Prepare the image file
File imageFile = new File(imagePath);
// Perform the POST request with the image as a part of the request body
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
body.add("file", new FileSystemResource(imageFile));
System.out.println("Image upload started "+id);
webClient.post().uri(url).contentType(MediaType.MULTIPART_FORM_DATA).body(BodyInserters.fromMultipartData
(body)).retrieve().bodyToMono(String.class).subscribe(response -> {
System.out.println("Response Id"+id+ ":" + response);
});
}
OutOfMemoryError
当我们在OpenJDK 11中运行这两个程序时,使用基于NIO的Spring WebClient的程序在几次迭代后出现了‘java.lang.OutOfMemoryError: Direct buffer memory'
,而基于Spring RestTemplate
的程序成功完成。以下是基于NIO的Spring WebClient程序的输出。您可以注意到报告了'java.lang.OutOfMemoryError'
。
Starting to post an image for Id0
Starting to post an image for Id1
Starting to post an image for Id2
Starting to post an image for Id3
Starting to post an image for Id4
Starting to post an image for Id5
Starting to post an image for Id6
Starting to post an image for Id7
Starting to post an image for Id8
Starting to post an image for Id9
Starting to post an image for Id10
Starting to post an image for Id11
Starting to post an image for Id12
Starting to post an image for Id13
Starting to post an image for Id14
2023-12-06 17:21:46.730 WARN 13804 --- [tor-http-nio-12] io.netty.util.concurrent.DefaultPromise : An
exception was thrown by reactor.ipc.netty.FutureMono$FutureSubscription.operationComplete()
reactor.core.Exceptions$ErrorCallbackNotImplemented:
io.netty.channel.socket.ChannelOutputShutdownException: Channel output shutdown
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.base/java.nio.Bits.reserveMemory(Bits.java:175) ~[na:na]
at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118) ~[na:na]
at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:318) ~[na:na]
at java.base/sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:242) ~[na:na]
at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:164) ~[na:na]
at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:130) ~[na:na]
at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:496) ~[na:na]
at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:418) ~[netty-
transport-4.1.23.Final.jar!/:4.1.23.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:934) ~[netty-
transport-4.1.23.Final.jar!/:4.1.23.Final]
... 18 common frames omitted
‘OutOfMemoryError: Direct buffer memory’
为了解决这个问题,我们利用了yCrash监控工具。该工具能够在问题在生产环境中出现之前预测到问题。一旦它在环境中预测到了问题,它会捕获360°环境中的故障排除工件,并对其进行分析,立即生成根本原因分析报告。它捕获的工件包括垃圾收集日志、线程转储、堆替代、netstat、vmstat、iostat、top、top -H、dmesg、内核参数和磁盘使用情况……。
您可以在此注册并开始使用该工具的免费版本。
yCrash服务器分析了Spring Boot Rest Client,并提供了清晰的问题指示和建议。以下是yCrash为SpringBoot WebClient应用程序生成的事件摘要报告。您可以注意到yCrash清楚地指出了错误,并提出了必要的建议以纠正问题。
yCrash的垃圾收集(GC)分析报告显示连续运行了完整的GC(如下面的截图所示)。当GC运行时,整个应用程序会暂停,不会处理任何事务。整个应用程序将变得无响应。我们观察到在SpringBoot WebClient应用程序崩溃之前,应用程序出现了无响应的行为,这是由于OutOfMemoryError
。
OutOfMemoryError
:直接缓冲区内存yCrash应用程序日志分析报告显示应用程序遭受‘java.lang.OutOfMemoryError: Direct buffer memory’
(见下面的截图) 导致应用程序崩溃。
OutOfMemoryError
? Spring WebClient基于Java NIO技术开发。在Java NIO中,对象存储在JVM本地内存的“直接缓冲区内存”区域,而RestTemplate对象存储在JVM本地内存的“other”区域。JVM中有不同的内存区域。要了解有关它们的信息,您可以观看这个视频剪辑。
当我们执行以上两个程序时,我们将直接缓冲区内存大小设置为200k(即-XX:MaxDirectMemorySize=200k)。这个大小对于Spring RestTemplate来说是足够的,因为对象从未存储在这个区域,但对于Spring WebClient来说不够。因此,Spring WebClient受到java.lang.OutOfMemoryError: Direct buffer memory
的影响。
在确定了这个问题之后,我们使用JVM参数-XX:MaxDirectMemorySize=1000k,将直接内存大小增加到了更高的值。做出这个改变后,Spring WebClient程序可以完美地运行而无任何问题。
Starting to post an image for Id0
Starting to post an image for Id1
Starting to post an image for Id2
Starting to post an image for Id3
Starting to post an image for Id4
Starting to post an image for Id5
Starting to post an image for Id6
Starting to post an image for Id7
Starting to post an image for Id8
Starting to post an image for Id9
Starting to post an image for Id10
Starting to post an image for Id11
Starting to post an image for Id12
Starting to post an image for Id13
Starting to post an image for Id14
Starting to post an image for Id15
Starting to post an image for Id16
Starting to post an image for Id17
Starting to post an image for Id18
Starting to post an image for Id19
Response Id11:Image uploaded successfully!
Response Id4:Image uploaded successfully!
Response Id1:Image uploaded successfully!
Response Id18:Image uploaded successfully!
Response Id2:Image uploaded successfully!
Response Id3:Image uploaded successfully!
Response Id6:Image uploaded successfully!
Response Id5:Image uploaded successfully!
Response Id10:Image uploaded successfully!
Response Id13:Image uploaded successfully!
Response Id15:Image uploaded successfully!
Response Id8:Image uploaded successfully!
Response Id17:Image uploaded successfully!
Response Id9:Image uploaded successfully!
Response Id7:Image uploaded successfully!
Response Id0:Image uploaded successfully!
Response Id16:Image uploaded successfully!
Response Id14:Image uploaded successfully!
Response Id19:Image uploaded successfully!
Response Id12:Image uploaded successfully!
在本文中,我们讨论了在从Spring RestTemplate
升级到基于Java NIO的WebClient时遇到的OutOfMemoryError问题。我们还分享了我们采取的诊断方法以及问题的解决方案。希望这对您有所帮助。
推荐阅读: 4.有10个文件,每个文件1G,每个文件的每一行存放的都是用户的query,每个文件的query都可能重复。要求你按照query的频度排序。
本文链接: Spring RestTemplate 到 WebClient 引起 OutOfMemoryError