Gradle Plugin: Classloader caching
Why you may want to cache classloaders
Before 1.7.0 the detekt Gradle plugin used the javaexec method to execute a detekt run.
This was practical as it started a new JVM and created an isolated environment.
However starting a new process for every sub module lead to a JVM start time overhead.
This is especially inconvenient performance-wise concerning large multi-project Gradle builds.
It also appears probable that processing a 40mb load of libraries all over again comes along with a start time penalty.
Reflection plus Classloader caching to the rescue
Instead of calling your application via javaexec you may also use reflection.
This may save us ~200ms when starting a JVM.
Unfortunately reflection alone however lead to metaspace OOM problems due to the size of the detekt-cli jar
if executing the detekt Gradle task in parallel with org.gradle.parallel=true enabled.
Even closing the classloader which was used for reflection didn’t help here as the garbage collector
was slower than Gradle opening more and more 40+ mb classloaders.
Opening and closing so many classloaders felt like a waste. After some hours of try-and-error and googling I found a GitHub discussion where someone from Gradle proposed caching the class loader (https://github.com/diffplug/spotless/issues/61).
This is the class loader cache I came up for detekt in the end:
package io.gitlab.arturbosch.detekt.internal
import org.codehaus.groovy.runtime.DefaultGroovyMethodsSupport
import org.gradle.api.file.FileCollection
import java.net.URLClassLoader
// It's important that the cache class is a singleton so it lives as long
// as a Gradle daemon lives.
object ClassLoaderCache {
private var loaderAndClasspath: Pair<URLClassLoader, FileCollection>? = null
// Gradle tasks can be run in parallel therefore we synchronize this function
fun getOrCreate(classpath: FileCollection): URLClassLoader = synchronized(ClassLoaderCache) {
val lastLoader = loaderAndClasspath?.first
val lastClasspath = loaderAndClasspath?.second
if (lastClasspath == null) {
cache(classpath)
} else {
// This is the central part of the caching.
// Whenever the classpath changes, we need to update our cache.
if (!lastClasspath.minus(classpath).isEmpty) {
// Do not forget to close the old classloader to avoid memory leaks!!!
DefaultGroovyMethodsSupport.closeQuietly(lastLoader)
cache(classpath)
}
}
return loaderAndClasspath?.first ?: error("Cached or newly created detekt classloader expected.")
}
// detekt is configured dynamically and uses Kotlin and it's embeddable compiler.
// We do not want to somehow use Gradle's versions of these jars.
private fun cache(classpath: FileCollection) {
loaderAndClasspath = URLClassLoader(
classpath.map { it.toURI().toURL() }.toTypedArray(),
null /* isolate detekt environment */
) to classpath
}
}
