I use the declarative Jenkins pipeline in my current project, but after 2 years of development we hit a barrier I wasn’t aware of. Jenkins puts the whole declarative pipeline into a single method and at a certain size the JVM fails with java.lang.RuntimeException: Method code too large!. Digging deeper it turns out that no method is allowed to exceed 64k. So I needed to find a solution for this problem.
The exact stacktrace I was presented with looked like this:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
General error during class generation: Method code too large!
java.lang.RuntimeException: Method code too large!
at groovyjarjarasm.asm.MethodWriter.a(Unknown Source)
[...]And I actually achieved it with a Jenkinsfile with roughly 1000 lines. The lines are no direct implication how large the resulting bytecode of the method will be, but it should make the point clear that this limit is not that far off in the context of a build pipeline.
There are already tickets in Jenkins’ Jira but there is no resolution on the horizon yet.
I see three solutions for the problem but all of them have their own implications:
Update 2020-01-22
I resolved the issue at last by transferring to the scripted pipeline and using a shared library that is located in a subfolder (See this post to figure out how to do that).
Use Shared Libraries
I already have a shared library in place for some complicated steps that we need to perform like getting the result of our Sonarqube analyses into the matching pull request in Bitbucket, but I have a major problem with this approach.
Shared Libraries, like the name suggests, should be shared and since they need to live in a separate repository you have a significant maintenance effort for strongly coupled features that are just for your very own project. So generally I dislike using them in their current form because of the extra maintenance effort.
Also this approach won’t scale endlessly since you can still reach the point that every stage is a one-liner and you can’t shorten them further.
| Pro | Con |
|---|---|
| No major refactoring needed | Hard to maintain because nearly every change needs to touch the shared library |
| Can be applied piece by piece | Harder to understand what exactly a step does |
| Resulting Jenkinsfile will be easy to read | Still limited, especially in a pipeline with many steps |
Put the Steps Into Methods outside of the Pipeline
This solution is currently an undocumented hack. You can since mid 2017 simply declare a method at the end of the pipeline and call it within you declarative pipeline. With this we achieve the same like we would with the shared library but we avoid the maintenance overhead.
| Pro | Con |
|---|---|
| No extra maintenance overhead | Undocumented and this solution might not work anymore at some point |
| All functionality reflected in the Jenkinsfile | Still limited, especially in a pipeline with many steps |
Migrate to Scripted Pipeline
As a last resort we can migrate to a scripted pipeline. With that we’ll have every freedom. But we’ll also loose the reason why we decided for the declarative pipeline in the first place. Having a dedicated DSL makes it easy to understand how the pipeline works and less error-prone in extending.
| Pro | Con |
|---|---|
| No limitation at all | Major refactoring necessary |
| More error-prone | |
| Probably more code needed to achieve the same functionality |
Conclusion
Since the exception occurred while developing a completely different feature I decided to use the workaround with an extra method at the end of the file for now. But its definite that we need to have a long term solution. So unless we find another solution we’ll aim to migrate to the scripted pipeline. But by doing so we’ll need to make sure that we implement a proper life cycle for our pipeline to avoid falling into a maintenance hell.