一、错误描述

jvm发生内存溢出“org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: Metaspace”错误，详细错误日志>>

org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: Metaspace@b@	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1075)@b@	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:962)@b@	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)@b@	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)@b@	at javax.servlet.http.HttpServlet.service(HttpServlet.java:665)@b@	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)@b@	at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)@b@	at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)@b@	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)@b@	at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:88)@b@	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)@b@	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)@b@	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)@b@	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)@b@	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)@b@	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119)@b@	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)@b@	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)@b@	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)@b@	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)@b@	at com.xwood.common.cloud.common.core.security.component.AfsUserInfoInjectFilter.doFilterInternal(AfsUserInfoInjectFilter.java:65)@b@	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)@b@	at org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:129)@b@	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)@b@	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)@b@	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)@b@	at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)@b@	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)@b@	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)@b@	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)@b@	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)@b@	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)@b@	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)@b@	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)@b@	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)@b@	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)@b@	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)@b@	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)@b@	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)@b@	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)@b@	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)@b@	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)@b@	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)@b@	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)@b@	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)@b@	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)@b@	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)@b@	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)@b@	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:93)@b@	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)@b@	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)@b@	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)@b@	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)@b@	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)@b@	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)@b@	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)@b@	at com.xwood.common.cloud.common.core.holder.RequestHeaderFilter.doFilter(RequestHeaderFilter.java:74)@b@	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)@b@	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)@b@	at com.xwood.common.cloud.common.core.security.filter.IgnoreUriFilter.doFilter(IgnoreUriFilter.java:48)@b@	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)@b@	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)@b@	at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)@b@	at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)@b@	at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)@b@	at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)@b@	at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68)@b@	at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117)@b@	at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)@b@	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)@b@	at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)@b@	at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)@b@	at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)@b@	at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)@b@	at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)@b@	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)@b@	at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52)@b@	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)@b@	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269)@b@	at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78)@b@	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133)@b@	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130)@b@	at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)@b@	at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)@b@	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249)@b@	at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78)@b@	at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99)@b@	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:387)@b@	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:841)@b@	at org.apache.skywalking.apm.plugin.undertow.v2x.SWRunnable.run(SWRunnable.java:45)@b@	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)@b@	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2019)@b@	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1558)@b@	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1449)@b@	at java.lang.Thread.run(Thread.java:748)@b@Caused by: java.lang.OutOfMemoryError: Metaspace@b@	at java.lang.ClassLoader.defineClass1(Native Method)@b@	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)@b@	at com.alibaba.fastjson.util.ASMClassLoader.defineClassPublic(ASMClassLoader.java:174)@b@	at com.alibaba.fastjson.serializer.ASMSerializerFactory.createJavaBeanSerializer(ASMSerializerFactory.java:409)@b@	at com.alibaba.fastjson.serializer.SerializeConfig.createASMSerializer(SerializeConfig.java:90)@b@	at com.alibaba.fastjson.serializer.SerializeConfig.createJavaBeanSerializer(SerializeConfig.java:252)@b@	at com.alibaba.fastjson.serializer.SerializeConfig.createJavaBeanSerializer(SerializeConfig.java:118)@b@	at com.alibaba.fastjson.serializer.SerializeConfig.getObjectWriter(SerializeConfig.java:821)@b@	at com.alibaba.fastjson.serializer.SerializeConfig.getObjectWriter(SerializeConfig.java:440)@b@	at com.alibaba.fastjson.serializer.JSONSerializer.getObjectWriter(JSONSerializer.java:448)@b@	at com.alibaba.fastjson.serializer.FieldSerializer.writeValue(FieldSerializer.java:208)@b@	at com.alibaba.fastjson.serializer.JavaBeanSerializer.write(JavaBeanSerializer.java:472)@b@	at com.alibaba.fastjson.serializer.JavaBeanSerializer.write(JavaBeanSerializer.java:154)@b@	at com.alibaba.fastjson.serializer.JSONSerializer.write(JSONSerializer.java:312)@b@	at com.alibaba.fastjson.JSON.toJSONString(JSON.java:793)@b@	at com.alibaba.fastjson.JSON.toJSONString(JSON.java:731)@b@	at com.alibaba.fastjson.JSON.toJSONString(JSON.java:688)

二、解决方法

在服务的启动JAVA_OPTS参数中将 MaxMetaspaceSize 最大元空间内存扩大（如从 128m 增大到 256m）

JAVA_OPTS="-server -XX:MetaspaceSize=64M -XX:MaxMetaspaceSize=256m   ...

其他更多Metaspace 参数汇总说明>>

参数	含义@b@-XX:MetaspaceSize=N	 初始化的 Metaspace 大小，该值越大触发 Metaspace 区域 GC 的时机就越晚。随着GC的到来，JVM 会根据实际情况调控 Metaspace 的大小，可能增加上限也可能降低@b@-XX:MaxMetaspaceSize=N	 限制 Metaspace 内存的最大值，防止因为某些情况导致 Metaspace 无限使用本地内存，影响到其他程序@b@-XX:MinMetaspaceFreeRatio=N	当进行过 Metaspace 区域 GC 之后，JVM 会计算当前 Metaspace 的空闲空间比，如果空闲比小于这个参数，那么 JVM 将增大 Metaspace 的内存大小。该参数的默认值为40，也就是40%。设置这个参数可以控制 Metaspace 的增长的速度，太小的值会导致 Metaspace 增长缓慢，Metaspace 的使用逐渐趋于饱和，可能会影响之后类的加载。而太大的值会导致 Metaspace 增长过快，浪费内存@b@-XX:MaxMetasaceFreeRatio=N	当进行过 Metaspace 区域的GC之后， 会计算当前 Metaspace 的空闲空间比，如果空闲比大于这个参数，那么虚拟机会释放 Metaspace 的部分空间。该参数的默认值为70，也就是70%@b@-XX:MaxMetaspaceExpansion=N	Metaspace 增长时的最大幅度，参数的默认值为5452592B（大约为5MB)@b@-XX:MinMetaspaceExpansion=N	Metaspace 增长时的最小幅度，该参数的默认值为340784B（大约为330KB）

总结，说明如下>>

1）引用的 jar 包加载了很多 class@b@这种情况需要排查服务中引用的比较大的 jar 包，去除掉多余的加载项，对于那些不能去掉的，按照最小粒度去加载@b@@b@2)动态生成类过多@b@Java 的许多框架会大量使用动态代理等技术动态生成类，使用这样的方式创建对象可能会有大量动态类产生，元空间消耗会增加，这种情况似乎没有好的解决方法