An Opinionated Guide to Modern Java, Part 2(blog.paralleluniverse.co)
blog.paralleluniverse.co
An Opinionated Guide to Modern Java, Part 2
http://blog.paralleluniverse.co/2014/05/08/modern-java-pt2/
17 comments
Nice article - Many of the suggestions (single-jar deployment, metrics, slf4j logging, etc) are all wrapped up for you in Dropwizard http://dropwizard.io, which we use and love.
Do you just use it for producing a REST API or do you use it to render HTML, handle session management etc.?
We use dropwizard only for rest APIs currently. Our HTML and JS are served up with an in-house framework that has plugins for less, minimization, svg inlining, etc. We have open tasks to both publish it and make it hostable in dropwizard directly.
We use it for both REST APIs on top our services and rendering templated HTML. We had to bastardize the asset bundle extension considerably for that though. Once we got things working we didn't want to go back.
I happen to use Dropwizard as well and purely for REST APIs that serve as internal micro services.
Sharing a lot of the Dropwizard philosophy, there is Spring Boot (http://projects.spring.io/spring-boot/), which we use and love as well.
I'm not clear, just did a ton of Googling and I think maybe spring.io is related to the Spring Framework at VMware. That would mean it's dependency injection.
Rather than downvote you, I thought I'd clear up the confusion. Yes Spring.io is the Spring Framework's website. Spring the framework is a dependency injection framework, but that's not all it does. There are other modules, like Spring Roo, Spring MVC, Spring JPA, etc that provide more than just dependency injection.
OK, I recalled it being a different looking site about three years ago then I saw Pivotal at the bottom of the page so I'm still not sure if VMware sold the Spring business or what. The whois reports for vmware.com and spring.io look nothing alike either. Thanks for clarifying spring.io is really Spring Framework.
VMware is owned by EMC. VMware bought Spring Source. EMC bought Pivotal Labs. VMware and EMC subsequently decided to spin-out a few of their acquisitions, including Spring and Pivotal Labs: into a company they dubbed Pivotal Software.
We'll discuss Dropwizard in part 3, but the single-jar deployment is not really solved by Dropwizard. To launch the server you still need to configure the JVM at the command line.
An off topic question; Would it be possible to have an RSS feed on your blog? I see you have Twitter but personally I don't really use that.
is that really a big issue? passing the arguments to java to configure the jvm.
It can be. Here's a typical example from my day to day:
java -cp "lib/*" -Xmx5g -Xms5g -server -XX:+UseTLAB -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:MaxTenuringThreshold=0 -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSParallelRemarkEnabled -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled
That gets cumbersome really quick.
Honestly my preferred jvm deployment is supervisord. I'd like a java only solution if one should exist that's good but these newer containers don't work for self made servers that don't run http.
Some of the instrumentation could be good, but at the end of the day I'm not sure how much it would help.
java -cp "lib/*" -Xmx5g -Xms5g -server -XX:+UseTLAB -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:MaxTenuringThreshold=0 -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSParallelRemarkEnabled -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled
That gets cumbersome really quick.
Honestly my preferred jvm deployment is supervisord. I'd like a java only solution if one should exist that's good but these newer containers don't work for self made servers that don't run http.
Some of the instrumentation could be good, but at the end of the day I'm not sure how much it would help.
Capsule works for any JVM application. It doesn't even have to be Java. It can use Java libraries, native libraries, a well-known server, your own server, with http, with a custom Infiniband thingy -- what have you.
I'll look in to it when there's a maven plugin maybe. As of right now my work flow works fine with the assembly plugin.
It looks pretty solid though. I'm still not totally sold on gradle as of yet mainly because of the tooling.
The native libs thing is a plus though. I'll see where I can work it in. Good work with capsule though.
It looks pretty solid though. I'm still not totally sold on gradle as of yet mainly because of the tooling.
The native libs thing is a plus though. I'll see where I can work it in. Good work with capsule though.
I'm glad someone took the time to write this, it's actually a very nice resource to get new coworkers up to speed. Shouldn't be tought as dogma of course (e.g. Gradle vs. Maven).
Also, thanks for mentioning Packr even though noone has used it in production yet. It's only a week old.
Assuming OP is the original author: are you by any chance following JGO? I see Quasar is actually using Matthias Mann's green thread lib which i don't think was advertised anywhere outside JGO.
Also, thanks for mentioning Packr even though noone has used it in production yet. It's only a week old.
Assuming OP is the original author: are you by any chance following JGO? I see Quasar is actually using Matthias Mann's green thread lib which i don't think was advertised anywhere outside JGO.
Yep. JGO. And Quasar now uses a heavily modified fork of Matthias Mann's code.
"Java application servers are dead" and since there's no alternative to Java application servers here's a solution I cooked up myself.
Like I mentioned last time I appreciate an overview of modern Java practices but boy howdy I can't discern if this is clever trolling or a cheap way to make me read Part 3.
Like I mentioned last time I appreciate an overview of modern Java practices but boy howdy I can't discern if this is clever trolling or a cheap way to make me read Part 3.
I skimmed through Part 3 and it's basically saying that app servers are hard to deploy, maintain and dev against and you should be using Spring Boot or Dropwizard instead to create applications with the services you require.
App servers like TomEE, JBoss and GlassFish are not hard to install at all, and certainly not to dev against. What's the author smoking?
These servers are just unzip == fully installed things. We have them checked in and just deploy them automatically whenever there's a need for an update, pretty much as every other library out there.
I wonder are we missing something or do people just wrongly think there's "something" difficult, where there's no need at all for things to be difficult?
These servers are just unzip == fully installed things. We have them checked in and just deploy them automatically whenever there's a need for an update, pretty much as every other library out there.
I wonder are we missing something or do people just wrongly think there's "something" difficult, where there's no need at all for things to be difficult?
Where did you find part 3?
Part 3 will be released toward the end of the week.
Ah my apologies, it's not actually part 3 it's the link he referenced when he talked about part 3: http://www.slideshare.net/ewolff/java-application-servers-ar...
I used Spring Boot the other day. It was really easy! Up until things just didn't work for no apparent reason, and neither the error messages nor documentation gave me any clues as to how to fix it.
App servers have historically been hard to deploy, maintain and dev against. There has been a huge amount of progress in the last few years. Based on my experience, Java EE / Wildfly is actually easier to develop with than Spring Boot.
App servers have historically been hard to deploy, maintain and dev against. There has been a huge amount of progress in the last few years. Based on my experience, Java EE / Wildfly is actually easier to develop with than Spring Boot.
I'm guessing tomcat.
I'd not go so far as to say it's trolling. Perhaps overly-dramatic hyperbole. As a Java developer, I do see a shift away from packaging up wars and deploying them to application servers. See Dropwizard or Spring Boot for examples of frameworks that prefer to run fat jars that serve http requests via embedded containers.
Why is the Java community so enamored with creating Java-only solutions when general-purpose solutions work just as well (and often better)?
There's no need to bundle everything up into an executable jar or war. It's much easier to write a Chef recipe to copy all the right files to the right places. And it's even easier to create a Docker container with the exploded war in its correct place. Both of those solutions don't rely on the clunky put-everything-inside-a-jar and the resulting need to write your own ClassLoader.
There's no need to bundle everything up into an executable jar or war. It's much easier to write a Chef recipe to copy all the right files to the right places. And it's even easier to create a Docker container with the exploded war in its correct place. Both of those solutions don't rely on the clunky put-everything-inside-a-jar and the resulting need to write your own ClassLoader.
1. Because the Java ecosystem usually gives you an API to everything: you can configure things to work exactly as you want. Everything is programmable and easily hackable.
2. Because everything is standardized, which is a lot easier than stitching together lots of pieces, each has to be deployed upgraded, configured, monitored and managed differently.
3. Because some Java libraries are used to do very sensitive stuff (performance wise, or correctness wise, or security wise), Java libraries, on average, tend to be of very high quality, at least relative to almost everything else (well, except maybe the Java browser plugin, but we're talking server-side here).
4. With Capsule there is no need to use a custom class loader, it's not any clunkier than Docker, and it's cross platform. It's just a cross-platform Java executable, which is stateless and requires no installation.
2. Because everything is standardized, which is a lot easier than stitching together lots of pieces, each has to be deployed upgraded, configured, monitored and managed differently.
3. Because some Java libraries are used to do very sensitive stuff (performance wise, or correctness wise, or security wise), Java libraries, on average, tend to be of very high quality, at least relative to almost everything else (well, except maybe the Java browser plugin, but we're talking server-side here).
4. With Capsule there is no need to use a custom class loader, it's not any clunkier than Docker, and it's cross platform. It's just a cross-platform Java executable, which is stateless and requires no installation.
I agree with using tools from other languages for the job. In our company, we chose python fabric. Wrote some deploy scripts with it.
It basically downloads the new version, updates the database running liquibase, updates some configuration files that are on git and finally updates the application war on tomcat.
Not so bad for a single file script.
It basically downloads the new version, updates the database running liquibase, updates some configuration files that are on git and finally updates the application war on tomcat.
Not so bad for a single file script.
Java - "run anywhere" (supposedly)
Docker - "The Linux Container Engine"
Think you've got your answer there.
Docker - "The Linux Container Engine"
Think you've got your answer there.
> Java - "run anywhere" (supposedly)
Sure, it's a nice benefit, but is it actually used? In 2014, how many of us are doing backend/service development in java and not deploying to Linux machines? There are certainly still backend/service java shops out there that aren't deploying to linux machines, but I wager that most are.
Regardless of those numbers though, why do the linux shops actually care about being able to deploy to non-linux machines if they never do in practice? Python can run just about anywhere these days, but python shops that deploy exclusively on Linux don't care about maintaining deployment compatibility with systems that they don't actually use.
Sure, it's a nice benefit, but is it actually used? In 2014, how many of us are doing backend/service development in java and not deploying to Linux machines? There are certainly still backend/service java shops out there that aren't deploying to linux machines, but I wager that most are.
Regardless of those numbers though, why do the linux shops actually care about being able to deploy to non-linux machines if they never do in practice? Python can run just about anywhere these days, but python shops that deploy exclusively on Linux don't care about maintaining deployment compatibility with systems that they don't actually use.
In 2014, how many of us are doing backend/service development in java and not deploying to Linux machines?
Among silicon valley webstartups, I would be surprised to hear much about java service/backend work at all. My experience is primarily with the megacorps.
If you are developing backend/service shit for other companies to use, then that is one thing, but I am thinking about in-house development. Most megacorps aren't software vendors; if they are developing software it is for themselves.
Regardless though, some megacorps deploying to windows doesn't explain the practices of java shops that don't deploy to windows. It seems like portability just for the sake of portability, without any real purpose but with real added hassle.
If you are developing backend/service shit for other companies to use, then that is one thing, but I am thinking about in-house development. Most megacorps aren't software vendors; if they are developing software it is for themselves.
Regardless though, some megacorps deploying to windows doesn't explain the practices of java shops that don't deploy to windows. It seems like portability just for the sake of portability, without any real purpose but with real added hassle.
You would be? Because its daily I hear about services written on the JVM.
GNU/Linux != UNIX, and there are still lots of it on the enterprise.
At the enterprise it is also common to develop on Windows and deploy across Windows and UNIX systems.
Finally, there are quite a few banks, finance and insurance institutions running Java on mainframes.
At the enterprise it is also common to develop on Windows and deploy across Windows and UNIX systems.
Finally, there are quite a few banks, finance and insurance institutions running Java on mainframes.
I've written telephony apps that deployed to Linux, Unix and Windows. I've written an OpenGL / Swing app on Windows and deployed it for Mac. This kind of stuff (a) does happen and (b) is valuable.
I don't understand how you believe that I was implying that nobody develops cross platform software.
I am talking about one specific kind of development: Service/backend development done in-house in a corporation that is not a software vendor. If you're doing Swing/OpenGL work, you're not doing the sort of work that I am talking about.
In my experience, while this sort of development may deploy to different nix's at different companies, within any particular company it tends to always deploy on only type of system. Most "mega-corps" are not software vendors; when they develop software they are developing it for themselves. This means that they control the entire stack, making portability fairly pointless.
When development in these organizations is done in other languages without such a "portability culture", not a second of thought is given to portability. The second a service is written in Java though, everyone starts jumping through hoops for something that they will never use.
I am talking about one specific kind of development: Service/backend development done in-house in a corporation that is not a software vendor. If you're doing Swing/OpenGL work, you're not doing the sort of work that I am talking about.
In my experience, while this sort of development may deploy to different nix's at different companies, within any particular company it tends to always deploy on only type of system. Most "mega-corps" are not software vendors; when they develop software they are developing it for themselves. This means that they control the entire stack, making portability fairly pointless.
When development in these organizations is done in other languages without such a "portability culture", not a second of thought is given to portability. The second a service is written in Java though, everyone starts jumping through hoops for something that they will never use.
Portability of Java helps you avoid the situation when your sysadmin tells you that he just upgraded a production server because the previous version of your distro reached its EOL and now your product does not start because libfoobar.so that you depend on is not included in the distro any more and he can't build it from source because the distro's default compiler also switched from gcc to clang.
Portability is important, even when you supposedly know your deployment platform.
Portability is important, even when you supposedly know your deployment platform.
Docker is an option if you're deploying on Linux, which is what the majority of server deployments use these days. Chef works on the rest of the Unixes, including OS X. If you're deploying Java on a Windows server, you're doing it wrong.
And the days of needing to use different development and production OSs are over...if you develop in Windows and deploy on some Unix variant, you should be using a VM.
I'm still not buying ClassLoader contortions as being in any way justified to support the single deployable artifact requirement. It's just not necessary and can lead to many subtle and hard-to-track-down issues as well as introducing unnecessary performance overhead.
And the days of needing to use different development and production OSs are over...if you develop in Windows and deploy on some Unix variant, you should be using a VM.
I'm still not buying ClassLoader contortions as being in any way justified to support the single deployable artifact requirement. It's just not necessary and can lead to many subtle and hard-to-track-down issues as well as introducing unnecessary performance overhead.
Docker is an option if you're deploying on Linux, which is what the majority of server deployments use these days. Chef works on the rest of the Unixes, including OS X. If you're deploying Java on a Windows server, you're doing it wrong.
What about OS/400, OS/390, MVS, etc?
Believe it or not, the world is a lot bigger than *NIX and Windows. Go track down a mid-sized manufacturing company in the midwest or the southeast, for example, and I'd almost bet you money they have apps running on an AS/400 (iSeries), or an S/38 or S/36 or something, if not a mainframe.
What about OS/400, OS/390, MVS, etc?
Believe it or not, the world is a lot bigger than *NIX and Windows. Go track down a mid-sized manufacturing company in the midwest or the southeast, for example, and I'd almost bet you money they have apps running on an AS/400 (iSeries), or an S/38 or S/36 or something, if not a mainframe.
> If you're deploying Java on a Windows server, you're doing it wrong.
You're assuming you have control over what the customer's servers are.
You're assuming you have control over what the customer's servers are.
Capsule has exactly 0 class-loader contortions (that's why I'm not too keen on One-Jar)
ClassLoader contortions are one of the cleaner ways to handle the problem, but they're not the only way.
You could:
- Create a fat jar, which necessitates bytecode transformations to prevent namespace collisions if you don't want to be sloppy. JarJar did this kinda of thing many years ago, but it seems hard to trust a process like that.
- Create what is essentially a self-extracting archive. This looks like the path Capsule took. This introduces unnecessary state and increases startup time. This is what servlet containers do, so reinventing that wheel seems like a particularly foolish decision. Not to mention that it appears that Capsule adds the additional step of pulling in all dependencies from an external server...slow deployment and an additional point of failure!
I still believe that some sort of VM image or container makes the most sense. Short of that, any system that supports sshd can be configured with Chef. The "solutions" I see coming out of the Java camp all seem like hacks by comparison.
You could:
- Create a fat jar, which necessitates bytecode transformations to prevent namespace collisions if you don't want to be sloppy. JarJar did this kinda of thing many years ago, but it seems hard to trust a process like that.
- Create what is essentially a self-extracting archive. This looks like the path Capsule took. This introduces unnecessary state and increases startup time. This is what servlet containers do, so reinventing that wheel seems like a particularly foolish decision. Not to mention that it appears that Capsule adds the additional step of pulling in all dependencies from an external server...slow deployment and an additional point of failure!
I still believe that some sort of VM image or container makes the most sense. Short of that, any system that supports sshd can be configured with Chef. The "solutions" I see coming out of the Java camp all seem like hacks by comparison.
The next release will include the option to resolve (potentially download) all dependencies without launching the app, if someone finds that useful. If dependencies are resolved, or are embedded, startup time is increased by 100ms or so.
It still feels like a pre-automation mindset...this is a step that should happen during CI, not application startup or even deployment.
The delay is going to be a lot more than 100ms if you use something like this when scaling elastically. In that situation, you need to go from automatically provisioned to running and accepting load in as short a timeframe as possible. Fully-baked machine images or Docker containers work for this use case, your solution doesn't.
Also, making your Maven repository a dependency of your production environment just seems like a bad idea. It creates an additional attack vector from a security standpoint and an additional component that can fail from a reliability standpoint.
The delay is going to be a lot more than 100ms if you use something like this when scaling elastically. In that situation, you need to go from automatically provisioned to running and accepting load in as short a timeframe as possible. Fully-baked machine images or Docker containers work for this use case, your solution doesn't.
Also, making your Maven repository a dependency of your production environment just seems like a bad idea. It creates an additional attack vector from a security standpoint and an additional component that can fail from a reliability standpoint.
Copying a file takes the same time, regardless of it's capsule of chef that is copying the file. Once capsule has cached the file, you can cut a VM image, and no copying will occur later.
> making your Maven repository a dependency of your production environment It becomes a dependency of your deployment process, which probably already depends on a maven repo. And a maven repo can be just a NAS drive, which is often reliable enough for use in production.
> making your Maven repository a dependency of your production environment It becomes a dependency of your deployment process, which probably already depends on a maven repo. And a maven repo can be just a NAS drive, which is often reliable enough for use in production.
Interesting, a lot of people that are really into CI point to the Java ecosystem as the best example of how to do it. People make the artifact of other language build pipelines a .deb or something similar to have basically the same artifact as a .jar/.war
Because Java-only solution to a complicated problem is normally simpler.
Few reasons contribute to that. One of them is exhaustive pure Java library ecosystem, which means that it's very unlikely that you'll run into intricacies of native library access on your host system. Another is excellent tooling, which means that when all your processes are Java processes, lots of operations-related tasks (process management & monitoring, log rotation, etc) get significantly easier.
More on that I wrote few weeks ago: http://www.mikhanov.com/2014/03/31/a-love-letter-to-java-363
Few reasons contribute to that. One of them is exhaustive pure Java library ecosystem, which means that it's very unlikely that you'll run into intricacies of native library access on your host system. Another is excellent tooling, which means that when all your processes are Java processes, lots of operations-related tasks (process management & monitoring, log rotation, etc) get significantly easier.
More on that I wrote few weeks ago: http://www.mikhanov.com/2014/03/31/a-love-letter-to-java-363
Indeed, pure Java crypto libraries (Bouncy Castle) were really nice a few weeks ago. When other languages/frameworks are wrapping or calling openssl, it's a really nice feeling to have a common crypto algorithms implemented in your languge/platform. Same goes for image encoding/decoding, database access, lots of examples.
Plus, there's the benefit of (essentially) standard builds, project layout, dependency management, and deployment strategies (jar or war). A java developer can walk into a new job and be productive right away.
Plus, there's the benefit of (essentially) standard builds, project layout, dependency management, and deployment strategies (jar or war). A java developer can walk into a new job and be productive right away.
play framework as well ( with no jsp dependency)
> "Java application servers are dead" and since there's no alternative to Java application servers here's a solution I cooked up myself.
The alternative are servlet containers, like Jetty or Tomcat. Jetty for example is very popular and really good and can be easily embedded inside an app for single JAR deployments, which is an awesome way to deploy an app btw.
The alternative are servlet containers, like Jetty or Tomcat. Jetty for example is very popular and really good and can be easily embedded inside an app for single JAR deployments, which is an awesome way to deploy an app btw.
Fair enough, it's a matter of terminology then, I guess?
Generally I think of Tomcat as being as much of an application server as something like JBoss is.
Granted, JBoss/Wildfly has a lot more enterprise-y features, but they both 'serve' web 'applications.'
Generally I think of Tomcat as being as much of an application server as something like JBoss is.
Granted, JBoss/Wildfly has a lot more enterprise-y features, but they both 'serve' web 'applications.'
I was talking about app/servlet containers (Tomcat, JBoss, WebSphere, etc.), vs embedded, single-app servers (Jetty, embedded Tomcat, and Dropwizard, which is essentially Jetty+Jersey+added goodies)
Dropwizard is a framework that delegates to Jetty the responsibility of serving requests by default, but you can host a Dropwizard app on whatever server you like.
In Java terminology, "application server" has actually started to mean servers capable of the full Java EE stack, which really means EJB and JMS. Jetty is not capable of those. Tomcat from what I know is not capable of those either. Both are targeting first and foremost the servlets API, which is pretty light and arguably good (well, at least since the latest Servlets 3.1, which finally adds asynchronous readings of requests).
Servlets is the piece of Java EE that I actually like - for everything else, there are third-party libraries and frameworks - though I've been working lately with Play framework, which comes with its own server and deployments on top of servlet containers is not officially supported, but that was primarily because it is a fully async framework and Servlets < 3.1 was inadequate for that, so things might change.
I also prefer embedding and the deployment of fat JARs to WARs. Makes things easier - a WAR implies that you need a management interface and a configured instance of your production server. A JAR implies that everything comes bundled in, configuration and all and you only have to copy and execute it directly, plus with embedding you have more fine-grained control. When I was using an embedded Jetty for example, a fine-tuned the shit out of its thread and connection pools, all from code.
In Java terminology, "application server" has actually started to mean servers capable of the full Java EE stack, which really means EJB and JMS. Jetty is not capable of those. Tomcat from what I know is not capable of those either. Both are targeting first and foremost the servlets API, which is pretty light and arguably good (well, at least since the latest Servlets 3.1, which finally adds asynchronous readings of requests).
Servlets is the piece of Java EE that I actually like - for everything else, there are third-party libraries and frameworks - though I've been working lately with Play framework, which comes with its own server and deployments on top of servlet containers is not officially supported, but that was primarily because it is a fully async framework and Servlets < 3.1 was inadequate for that, so things might change.
I also prefer embedding and the deployment of fat JARs to WARs. Makes things easier - a WAR implies that you need a management interface and a configured instance of your production server. A JAR implies that everything comes bundled in, configuration and all and you only have to copy and execute it directly, plus with embedding you have more fine-grained control. When I was using an embedded Jetty for example, a fine-tuned the shit out of its thread and connection pools, all from code.
Thinking Java EE is primarily about EJB and JMS is pretty old school.
These days CDI is the centrepiece of Java EE, along with services like Interceptors and Bean Validation. JPA, JSF and JAX-RS also play pretty important roles, which are all things you don't find by default in Tomcat.
Actually, nearly every time I see people using Tomcat they add many of the things mentioned above. You might as well start with TomEE then and work from there.
These days CDI is the centrepiece of Java EE, along with services like Interceptors and Bean Validation. JPA, JSF and JAX-RS also play pretty important roles, which are all things you don't find by default in Tomcat.
Actually, nearly every time I see people using Tomcat they add many of the things mentioned above. You might as well start with TomEE then and work from there.
CDI, Interceptors, Bean Validation, JPA, JSF - all of them suck.
JAX-RS is the only thing OK-ish in the list you mentioned, unfortunately its design that ignored asynchronicity has been shortsighted (heard that it got fixed in 2.0, but apparently they also added junk).
JAX-RS is the only thing OK-ish in the list you mentioned, unfortunately its design that ignored asynchronicity has been shortsighted (heard that it got fixed in 2.0, but apparently they also added junk).
I like the overview given to JVM tooling, many developers are fully unaware of what JVMs (not only the official one from Oracle) offer in terms of monitoring.
If you want to go really low level, a few of them even show the generated assembly code by the JIT.
If you want to go really low level, a few of them even show the generated assembly code by the JIT.
You can do this from the JVM command line with "-XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly". (it's kind of a mess, though)
Yes, it still requires the plugins which aren't delivered with the JDKs and each JVM does it a bit differently, but is possible.
As complement of it, check JIT Watch
https://skillsmatter.com/skillscasts/5243-chris-newland-hots...
As complement of it, check JIT Watch
https://skillsmatter.com/skillscasts/5243-chris-newland-hots...
Oh, I forgot to mention JIT Watch! (I actually use it sometimes). I'll give it a quick mention next time.
As a Java developer, I thought this article had some interesting information about logging and monitoring. However, the deployment section had my scratching my head a little.
I've never totally understood why people want to make fat jars. It seems like a process full of headaches since you can't have jars in jars. Wouldn't it be much easier to create a regular zip file with a small script to set the classpath and and run the project?
I'm not sure I understand the motivation for embedded instead of standalone servlet container. The article linked to some slides but they mostly seemed be demonstrating that you can function using an embedded container rather than providing clear benefits. Maybe it would have made more sense with the associated talk.
Can anyone provide more insight to these?
I've never totally understood why people want to make fat jars. It seems like a process full of headaches since you can't have jars in jars. Wouldn't it be much easier to create a regular zip file with a small script to set the classpath and and run the project?
I'm not sure I understand the motivation for embedded instead of standalone servlet container. The article linked to some slides but they mostly seemed be demonstrating that you can function using an embedded container rather than providing clear benefits. Maybe it would have made more sense with the associated talk.
Can anyone provide more insight to these?
Actually, you can have Jars in Jars. Of course, you need to do the lifting yourself. It is actually very nice to have a simple unpack / installation application inside a Jar and to unpack the actual application from this Jar. In this way, the only requirement for unpack / installation is the JVM, which you already require to run the application itself. This is much more elegant than a platform dependent unpack / installation (installer binary or ZIP plus Windows Batch or a Shell script).
Huh, jars within jars... sounds a lot like .war files!
spring boot has done the lifting:
http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/refe...
That is actually very similar to Capsule, except that it doesn't support Maven, automatic upgrades, and as far as I can tell - JVM configuration (heap size, agents etc). I think Capsule is a strict superset in terms of features.
Author here. A capsule is not necessarily a fat jar. It can point to Maven dependencies that are downloaded on the first launch, and can later be shared by other capsules. A zip with startup scripts is OK, but it requires installation.
As to full blown app servers vs embedded servers, I think it's the other way around. It's the big app servers that require justification, as they are a lot more cumbersome to set up and deploy.
As to full blown app servers vs embedded servers, I think it's the other way around. It's the big app servers that require justification, as they are a lot more cumbersome to set up and deploy.
It can point to Maven dependencies that are downloaded on the first launch
You wouldn't do this for a production deployment, right? Application starup that may or may not require access to the artifact repository to complete successfully. When that idea bounces around my developer neocortex, my sysadmin hindbrain starts reaching forward to strangle it.
And if you're not going to do it in production, doing it in development means having a gratuitous difference between development and production, which, again, is something i have learned to fear.
A zip with startup scripts is OK, but it requires installation.
'gradle installApp' works out of the box, and 'installs' the jars and scripts in your build directory, which is all you need to run locally. It's work of minutes to write an fpm [1] invocation that packages the output of installApp as an operating system package, which you can then send down the pipe towards production. This is simple, easy, standard, and fully integrated with a world of existing package management tools. Why would i use Capsule instead of doing this?
[1] https://github.com/jordansissel/fpm
You wouldn't do this for a production deployment, right? Application starup that may or may not require access to the artifact repository to complete successfully. When that idea bounces around my developer neocortex, my sysadmin hindbrain starts reaching forward to strangle it.
And if you're not going to do it in production, doing it in development means having a gratuitous difference between development and production, which, again, is something i have learned to fear.
A zip with startup scripts is OK, but it requires installation.
'gradle installApp' works out of the box, and 'installs' the jars and scripts in your build directory, which is all you need to run locally. It's work of minutes to write an fpm [1] invocation that packages the output of installApp as an operating system package, which you can then send down the pipe towards production. This is simple, easy, standard, and fully integrated with a world of existing package management tools. Why would i use Capsule instead of doing this?
[1] https://github.com/jordansissel/fpm
Well, you can choose to embed the dependencies in the capsule, but I think the best approach for production deployment is have an organizational Maven repository (Artifactory/Nexus). This way you upload the libs, and the jars containing the actual apps to your repo, and all you need to do is restart the capsule (it can be configured to load and run the latest version).
So you're downloading jars from the repository to the production server when the app starts? I would feel very uneasy about that kind of coupling.
And i still don't see what advantage this has over just pushing out normal packages.
And i still don't see what advantage this has over just pushing out normal packages.
I feel uneasy about deploying the wrong version. With capsule, at launch it checks for a new version in the repo (if you configure it to use the newest version rather than a specific one). The packages are only downloaded once: not on every restart.
Alternatively, you can embed the dependencies, in which case it's just like a "normal package", only it doesn't require installation, and is just as easy to create (your build tool can create it). So it's a platform independent "normal package", with added features.
Alternatively, you can embed the dependencies, in which case it's just like a "normal package", only it doesn't require installation, and is just as easy to create (your build tool can create it). So it's a platform independent "normal package", with added features.
Interesting. I feel much more confident about deploying the right thing using an operating system package than some other mechanism. Almost everything else in the datacentre is deployed using operating system packages, so we get a lot of practice at deploying the right versions of things. The few legacy applications we have that are deployed via custom mechanisms are a headache - they require completely different tooling and troubleshooting knowledge to everything else.
But then, i have spent a fair amount of managing machines, shuffling packages around apt repositories, writing Puppet code and so on. Perhaps for a developer who has not served a sentence in operations, operating system packages are a less comforting proposition.
You seem to be very keen on avoiding "installation". Could you tell us about why that is?
And could you remind me what the added features of a capsule are? Putting aside the differences in delivery mechanism, which as i've said, i'm afraid i see as misfeatures, the only one i see is that it automatically finds the right JRE.
(Sorry you're being downvoted, by the way. I think this is an interesting discussion, and your gracious responses deserve upvotes, not downvotes.)
But then, i have spent a fair amount of managing machines, shuffling packages around apt repositories, writing Puppet code and so on. Perhaps for a developer who has not served a sentence in operations, operating system packages are a less comforting proposition.
You seem to be very keen on avoiding "installation". Could you tell us about why that is?
And could you remind me what the added features of a capsule are? Putting aside the differences in delivery mechanism, which as i've said, i'm afraid i see as misfeatures, the only one i see is that it automatically finds the right JRE.
(Sorry you're being downvoted, by the way. I think this is an interesting discussion, and your gracious responses deserve upvotes, not downvotes.)
The advantage Capsule gives you is statelessness. The user does not need to put her system in a certain state. You get one file and you run it. It should work regardless of anything else that's installed -- or not -- on your machine, with the exception of an OS and a JRE.
There are other ways to achieve stateless deployment, and Capsule is not always the right fit. For example, it's not the right fit if your app requires an Oracle DB installed locally (it could be done, because Capsule supports an installation script embedded in the JAR, but that probably wouldn't be a good idea in this case). But when it is the right fit (e.g. microservices, grid workers etc.), it's much more lightweight than any other solution.
There are other ways to achieve stateless deployment, and Capsule is not always the right fit. For example, it's not the right fit if your app requires an Oracle DB installed locally (it could be done, because Capsule supports an installation script embedded in the JAR, but that probably wouldn't be a good idea in this case). But when it is the right fit (e.g. microservices, grid workers etc.), it's much more lightweight than any other solution.
So your production application can potentially run in an untested configuration because someone has pushed a new version of XyzLib.
Again, for what advantage?
I mean, fine if you want to do that. But I wouldn't call it Modern Java or recommend anyone else do it.
Again, for what advantage?
I mean, fine if you want to do that. But I wouldn't call it Modern Java or recommend anyone else do it.
How is that any more dangerous from deploying and installing an OS package? It's very hard to accidentally deploy to a release Maven repo. Maven Central makes you jump through hoops, and organizational repos have their own safeguards. They already deploy everything to their Maven repo anyway, why deploy again?
And if you don't want to enable automatic upgrades, you still get stateless, installation-free deployment.
And if you don't want to enable automatic upgrades, you still get stateless, installation-free deployment.
That's the way we do it in production and it works great. Why create fat jar files and copy them to dozens of servers when you can just have each server pull down its dependencies from your repository.
So now the availability of your production service relies on the availability of your (development) repository. Another (pointless) point of failure.
And you can't see why this is a bad idea?
And you can't see why this is a bad idea?
It should be obvious that if the repository were unavailable, we would not try to push a new version of our code. This is a much better approach since library dependencies are only usually downloaded once when we start a service for the first time, vs a fat jar that pushes dependencies every single time, leading to long startup times. This is a big deal when you've got a hundred servers and dozens of services on each.
It can point to Maven dependencies that are downloaded on the first launch
Please don't. Never assume that production machines have access to the internet or even an internal Maven repository. If they do, they're a security risk. There are plenty of scenarios I can think of where an attacker could possibly make your application download something from the internet, or has your application server functioning as an anonymous proxy by exploiting your application.
Please don't. Never assume that production machines have access to the internet or even an internal Maven repository. If they do, they're a security risk. There are plenty of scenarios I can think of where an attacker could possibly make your application download something from the internet, or has your application server functioning as an anonymous proxy by exploiting your application.
Looks like I should have read the Capsule Github page. I didn't realize Capsule was caching the project dependencies. I could certainly see that making projects much faster to deploy.
Aren't the app servers and the embedded servers the same servers just configured differently? Standalone it is configured with XML and and embedded it is configured with Java code. It doesn't seem to me like there is much of a set up difference.
As far as I can tell, it comes down who cares more about the settings for the server, the people who maintain the machines or the people who maintain the code. If those people are the same then it sounds like it really doesn't matter.
Aren't the app servers and the embedded servers the same servers just configured differently? Standalone it is configured with XML and and embedded it is configured with Java code. It doesn't seem to me like there is much of a set up difference.
As far as I can tell, it comes down who cares more about the settings for the server, the people who maintain the machines or the people who maintain the code. If those people are the same then it sounds like it really doesn't matter.
Multi-app servers usually take much longer to launch, deploying apps and upgrading the servers is a pain, and there's a host of other issues (like not being able to easily allocate resources to apps according to their needs).
> As far as I can tell, it comes down who cares more about the settings for the server
There's a lot of truth to that. I think that the whole devops approach is that they should be the same people. Of course, you might have valid reasons for not subscribing to that approach.
> As far as I can tell, it comes down who cares more about the settings for the server
There's a lot of truth to that. I think that the whole devops approach is that they should be the same people. Of course, you might have valid reasons for not subscribing to that approach.
The difficulties you mention (aside from upgrading servers) is about the having multiple apps in a single server not standalone vs embedded. Although, if you are careful, it can actually be faster and easier to deploy to a multi-app server [0].
I grant you, upgrading servers can be more involved depending on how you do it.
[0] - http://tomcat.apache.org/tomcat-7.0-doc/deployer-howto.html#...
I grant you, upgrading servers can be more involved depending on how you do it.
[0] - http://tomcat.apache.org/tomcat-7.0-doc/deployer-howto.html#...
unzipping a folder and running a <10 line bash script requires installation? is that what you mean by installation?
Well, if it works for you then great. But doing that with every version (Capsule gives you automatic upgrades) might become annoying, and even dangerous, and if you can do the same thing with a single file that doesn't even require installation, why not make life even easier.
Ok, not disagreeing, just wondering what you meant by installation, Capsule sure looks cool.
Having moved from Tomcat to embedded Jetty, it avoids a bunch of problems - no two linux distributions can agree on where to put Tomcat particularly when you need multiple instances, and they have an unfortunate tendency to run out of PermGen space and need restarting after you've done a few redeploys. But it's mainly just faster, and much easier to use in development; you can just run the same java class either way, so running it with a debugger is easy, hot code replace just works, etc. As a bonus you can avoid having an XML config file. It's not a huge difference but it makes life slightly easier.
Fully agreed with the zip and script though, that's what we do (using the appassembler maven plugin, so it just happens as part of the build).
Fully agreed with the zip and script though, that's what we do (using the appassembler maven plugin, so it just happens as part of the build).
These arguments don't make sense to me.
I'm not sure why it matters that different distros put Tomcat in different locations. Embedding Jetty just means that now you have to restart every time. You could just do the same thing with standalone Tomcat instance. The only reason you are running out PermGen space on the redeploys is because your application is not cleaning up its threads on shutdown.
Why is it difficult to do any of those development things without having the server embedded in your application? Pretty much every Java build tool (Maven, SBT, Gradle, Lein) makes those things easy to do without specifying an XML file for your server. Also, they will frequently let you reload your application, which is faster.
I'm not sure why it matters that different distros put Tomcat in different locations. Embedding Jetty just means that now you have to restart every time. You could just do the same thing with standalone Tomcat instance. The only reason you are running out PermGen space on the redeploys is because your application is not cleaning up its threads on shutdown.
Why is it difficult to do any of those development things without having the server embedded in your application? Pretty much every Java build tool (Maven, SBT, Gradle, Lein) makes those things easy to do without specifying an XML file for your server. Also, they will frequently let you reload your application, which is faster.
> Embedding Jetty just means that now you have to restart every time. You could just do the same thing with standalone Tomcat instance.
So now either I need to figure out how to hook my deployment script into the host system's init scripts, or I need to package tomcat as part of my app (and worry about its shared library linkage). With embedded, my app is just a program, not at all integrated with the host system; it runs like a program and stops like a program.
> Why is it difficult to do any of those development things without having the server embedded in your application?
Because the tool needs support for the server and often it has to be done in a server-specific way. How does one use JProfiler on an app deployed in Tomcat? One edits one of the internal Tomcat xml files. How does one use JProfiler on an app that embeds Jetty? One runs it with the same command line arguments one uses for running any other app with JProfiler. And it's like that for every tool - the way to do it with Tomcat is slightly different, the way to do it with an embedded server is standard.
> Pretty much every Java build tool (Maven, SBT, Gradle, Lein) makes those things easy to do without specifying an XML file for your server.
If you're building a war you need a web.xml (or I'm seeing some speculation it can be done with annotations? Either way, much less clear than just making a servlet in code).
> Also, they will frequently let you reload your application, which is faster.
Not in a way that matters. If you're just doing day-to-day dev then IDE hot code replace is faster still. If you want to be sure you've cleaned out all traces of the old version and are definitely running your latest code then reload isn't reliable enough.
So now either I need to figure out how to hook my deployment script into the host system's init scripts, or I need to package tomcat as part of my app (and worry about its shared library linkage). With embedded, my app is just a program, not at all integrated with the host system; it runs like a program and stops like a program.
> Why is it difficult to do any of those development things without having the server embedded in your application?
Because the tool needs support for the server and often it has to be done in a server-specific way. How does one use JProfiler on an app deployed in Tomcat? One edits one of the internal Tomcat xml files. How does one use JProfiler on an app that embeds Jetty? One runs it with the same command line arguments one uses for running any other app with JProfiler. And it's like that for every tool - the way to do it with Tomcat is slightly different, the way to do it with an embedded server is standard.
> Pretty much every Java build tool (Maven, SBT, Gradle, Lein) makes those things easy to do without specifying an XML file for your server.
If you're building a war you need a web.xml (or I'm seeing some speculation it can be done with annotations? Either way, much less clear than just making a servlet in code).
> Also, they will frequently let you reload your application, which is faster.
Not in a way that matters. If you're just doing day-to-day dev then IDE hot code replace is faster still. If you want to be sure you've cleaned out all traces of the old version and are definitely running your latest code then reload isn't reliable enough.
Why does standalone Tomcat need any more integration with the system than an app with embedded Tomcat? They are both Java applications.
Tool support is not server specific. In both cases you set the appropriate VM arguments. What XML file would you have to edit?
Yes, hot code replacement is faster (although it comes with other tradeoffs). Why can't you do that with standalone Tomcat? What is unreliable about reloading? You can leak PermGen space if you don't finalize your threads properly, but why wouldn't you be running the latest code?
You're right about the web.xml file. I had forgotten about that. I can see why you might prefer to write that in Java code rather than XML.
Tool support is not server specific. In both cases you set the appropriate VM arguments. What XML file would you have to edit?
Yes, hot code replacement is faster (although it comes with other tradeoffs). Why can't you do that with standalone Tomcat? What is unreliable about reloading? You can leak PermGen space if you don't finalize your threads properly, but why wouldn't you be running the latest code?
You're right about the web.xml file. I had forgotten about that. I can see why you might prefer to write that in Java code rather than XML.
> Why does standalone Tomcat need any more integration with the system than an app with embedded Tomcat? They are both Java applications.
With an embedded Jetty my app can just be a jar or set of jars. Tomcat has a directory structure and ships as a binary rather than a jar so I'd need to unpack it and worry about it being linked correctly for the target system.
> Tool support is not server specific. In both cases you set the appropriate VM arguments. What XML file would you have to edit?
I forget the name - catalina.xml or something? You can't just run ./tomcat -agent:myagent.jar. because it won't pass the arguments through to the JVM, unless that's changed.
With an embedded Jetty my app can just be a jar or set of jars. Tomcat has a directory structure and ships as a binary rather than a jar so I'd need to unpack it and worry about it being linked correctly for the target system.
> Tool support is not server specific. In both cases you set the appropriate VM arguments. What XML file would you have to edit?
I forget the name - catalina.xml or something? You can't just run ./tomcat -agent:myagent.jar. because it won't pass the arguments through to the JVM, unless that's changed.
Tomcat is just a Java application. Its install is just a bunch of jar files with some shell scripts to simplify running them. If you look at an install you see that it is just running the Java class ' org.apache.catalina.startup.Bootstrap'. I'm not sure what linking you are worrying about.
For setting JVM arguments, I think you are thinking of 'setenv.sh' or 'setenv.bat'. These are shell scripts where you can set environment variables used when running Tomcat. This includes the standard 'JAVA_OPTS' to set VM arguments.
For setting JVM arguments, I think you are thinking of 'setenv.sh' or 'setenv.bat'. These are shell scripts where you can set environment variables used when running Tomcat. This includes the standard 'JAVA_OPTS' to set VM arguments.
> While I personally prefer Gradle’s nice DSL and the ability to use imperative code for non-common build operations, I can understand the preference for the fully declarative Maven, even if it requires lots of plugins. The modern Java developer, then, might prefer Maven to Gradle.
I find Maven's abstractions surprisingly hard to understand. It is a major part of my annoyance with Java as a dev setup. I miss Make/Ant. Not sure if Gradle will be a good replacement or not but would be curious to try.
I find Maven's abstractions surprisingly hard to understand. It is a major part of my annoyance with Java as a dev setup. I miss Make/Ant. Not sure if Gradle will be a good replacement or not but would be curious to try.
I find all of them hard to work with, because they are all aiming for the modern day "zero configuration" nirvana that Ruby on Rails achieved, and thus enormous amounts of implicit knowledge is required to understand them.
Part of the problem is that the initial steps are so easy that you're actually never forced to learn the mechanics of what is happening. Hence you get away with a lot until you need to do something there isn't a magic command for. This is also why you see blithe statements that you "don't need to know Groovy to use Gradle", etc. However these are only said after the fact when you've learned Gradle, understood enough Groovy to intuit what it's doing and then retrospectively realised there is a theoretical path from ignorance to enlightenment that didn't involve learning Groovy.
What you would probably like is Gant, which is Ant entirely converted to Groovy. It's missing all the higher level project stuff from Maven / Gradle, but it works a treat when you just want the simplest possible thing that could work.
Part of the problem is that the initial steps are so easy that you're actually never forced to learn the mechanics of what is happening. Hence you get away with a lot until you need to do something there isn't a magic command for. This is also why you see blithe statements that you "don't need to know Groovy to use Gradle", etc. However these are only said after the fact when you've learned Gradle, understood enough Groovy to intuit what it's doing and then retrospectively realised there is a theoretical path from ignorance to enlightenment that didn't involve learning Groovy.
What you would probably like is Gant, which is Ant entirely converted to Groovy. It's missing all the higher level project stuff from Maven / Gradle, but it works a treat when you just want the simplest possible thing that could work.
> I find all of them hard to work with, because they are all aiming for the modern day "zero configuration" nirvana that Ruby on Rails achieved, and thus enormous amounts of implicit knowledge is required to understand them.
> ... realised there is a theoretical path from ignorance to enlightenment that didn't involve learning Groovy.
Yes! Learning is so non-linear that sure there are in theory fine lines that minimize the amount of new stuff to learn. Yet, it never is that simple. Bad design, indeed.
> What you would probably like is Gant, which is Ant entirely converted to Groovy. It's missing all the higher level project stuff from Maven / Gradle, but it works a treat when you just want the simplest possible thing that could work.
I will check it out; thank you!
> ... realised there is a theoretical path from ignorance to enlightenment that didn't involve learning Groovy.
Yes! Learning is so non-linear that sure there are in theory fine lines that minimize the amount of new stuff to learn. Yet, it never is that simple. Bad design, indeed.
> What you would probably like is Gant, which is Ant entirely converted to Groovy. It's missing all the higher level project stuff from Maven / Gradle, but it works a treat when you just want the simplest possible thing that could work.
I will check it out; thank you!
At least with Maven you have a standardized build process in every project. Checkout the code and be productive right away. It structures dependencies and project layout.
Currently I'm in a large enterprise project where the build department decided it was a good idea to reimplement a whole complex build in gradle (basically all that maven does by default). It didn't help that they didn't realize Gradle uses Maven conventions by default, ignored that and ran their own dependency system. The build is REALLY slow and really complex and nobody has any idea exactly what it does.
In ant having a non-standard project layout was ok, but then at least the build was FAST and didn't take half a minute just to START the build system.
I'm not yet sure if gradle actually IS better if you use it right, or if it's just a very very slow solution to a problem you shouldn't have it the first place if you do it right with something like maven.
Currently I'm in a large enterprise project where the build department decided it was a good idea to reimplement a whole complex build in gradle (basically all that maven does by default). It didn't help that they didn't realize Gradle uses Maven conventions by default, ignored that and ran their own dependency system. The build is REALLY slow and really complex and nobody has any idea exactly what it does.
In ant having a non-standard project layout was ok, but then at least the build was FAST and didn't take half a minute just to START the build system.
I'm not yet sure if gradle actually IS better if you use it right, or if it's just a very very slow solution to a problem you shouldn't have it the first place if you do it right with something like maven.
I really enjoy Gradle. I have done significant work with Ant, Maven, and a host of tool chains in the C/C++ world. Gradle seems to have the cleanest DSL. Not every single thing under the sun is supported (Maven seems to be the most comprehensive tool I have used), but Gradle is nice to work with.
I've been struggling to find out what's even available in Gradle. How does one debug it, or print available methods, or really anything?
My woes are mostly around Android's gradle integration, which appears poorly documented at best. There are tons of magic commands that appear out of nowhere and with no explanation beyond "use this.". And Gradle's massive guide covers things that barely see the light after being seemingly wrapped in several layers by Android's plugin.
I want to give it a real try, but the Android projects I deal with are reasonably complex, would use code generation (`scopes.PROVIDED.plus += configurations.provided }`? wtf does that even do and how in the world would I have discovered I could use it?), and need multiple products. I would have to know what and why before considering switching over for real, but there seem to be some huge walls preventing me from finding out.
My woes are mostly around Android's gradle integration, which appears poorly documented at best. There are tons of magic commands that appear out of nowhere and with no explanation beyond "use this.". And Gradle's massive guide covers things that barely see the light after being seemingly wrapped in several layers by Android's plugin.
I want to give it a real try, but the Android projects I deal with are reasonably complex, would use code generation (`scopes.PROVIDED.plus += configurations.provided }`? wtf does that even do and how in the world would I have discovered I could use it?), and need multiple products. I would have to know what and why before considering switching over for real, but there seem to be some huge walls preventing me from finding out.
What abstractions? You have a project which has a name and some other metadata and it has a bunch of dependencies. If your build does something weird then it also has plugins, but you should try to avoid those as much as possible. I guess profiles could be confusing, but again, try to avoid them. Maven is the simplest build tool I've used, in any language.
Maven has a lot of jargon. I also found it confusing when I started. POMS, artifacts, plugins, reactors etc ... heck I still find Maven confusing and unintuitive sometimes. Their docs are awful: typical failure mode for software that's in love with the idea of plugins.
Exactly. With Make, I can delve down into the bash code easily and figure out what is happening. With Maven, there is enough overhead that I have to resort to cargo-cult googling.
> I personally prefer Gradle’s nice DSL [...] in order to use Gradle one does not need to know Groovy, even if one wishes to do some non-standard stuff [...] I just learned a few useful Groovy expressions that I found in Gradle examples online
The DSL is Groovy syntax from Groovy's antiquated Antlr 2.7 grammar, so simply by using Gradle you're using Groovy along with all its warts. Underneath, Gradle isn't so much a DSL as an API shipping with a programming language. You could just as easily write the first Gradle example from the article in most other JVM languages. If it was in Clojure...
The DSL is Groovy syntax from Groovy's antiquated Antlr 2.7 grammar, so simply by using Gradle you're using Groovy along with all its warts. Underneath, Gradle isn't so much a DSL as an API shipping with a programming language. You could just as easily write the first Gradle example from the article in most other JVM languages. If it was in Clojure...
(require gradle :as g)
(g/apply :plugin "java")
(g/apply :plugin "application")
(g/source-compatibility "1.8")
(g/main-class-name "jmodern.Main")
(g/repositories
(g/maven-central))
(g/configurations
g/quasar)
(g/dependencies
(g/compile "co.paralleluniverse:quasar-core:0.5.0:jdk8")
(g/compile "co.paralleluniverse:quasar-actors:0.5.0")
(g/quasar "co.paralleluniverse:quasar-core:0.5.0:jdk8")
(g/test-compile "junit:junit:4.11"))
(g/run
(g/jvm-args (str "-javaagent:" (-> (configurations.quasar.iterator) next))))Does anyone actually swap out their log backend? To me slf4j felt like an overcomplicated, enterprisey step with no clear advantage over log4j.
Think slf4j as API, and logj4 as implementation. slf4j has multiple implementations, e.g. logj4, logback, apache common logging bridge, so you can swap implementation easily without change your code(except configuration)
It's great because you can use whatever backend you're comfortable with in your setup. Are you a JUL guy? Great, configure JUL then.
I took a totally baroque project setup (they just don't know WTH they're doing) with some JUL and some log4j, piped everything into SLF via the bridges, then configured a log4j backend (which I'm confortable with) and now I'm the only person in the project that actually has control over their logging and configures it to log into nicely separated files for application output, SQL statements and so on :-)
In an ideal world we'd all just use one log system, but Sun screwed that when they created the very unnecessary JUL back in the day. SLF solves the logging mess nicely.
I took a totally baroque project setup (they just don't know WTH they're doing) with some JUL and some log4j, piped everything into SLF via the bridges, then configured a log4j backend (which I'm confortable with) and now I'm the only person in the project that actually has control over their logging and configures it to log into nicely separated files for application output, SQL statements and so on :-)
In an ideal world we'd all just use one log system, but Sun screwed that when they created the very unnecessary JUL back in the day. SLF solves the logging mess nicely.
SLF4j is already the standard wrt logging. I think one thing to think of here is libraries.
If there is one logging engine to depend on that libs can use, this means less dependency hell.
Fighting it is a losing battle with how much traction it has among all the different libs.
If there is one logging engine to depend on that libs can use, this means less dependency hell.
Fighting it is a losing battle with how much traction it has among all the different libs.
I just switched from Logback to Log4j2 for an application and since I was using SLF4J, there were no code changes required.
Also consider the case of third-party libraries. I have dependencies in my application that are hard coded to log to about three different logging implementations. If they instead used SLF4J, the end user could choose whichever implementation they were already using in their app, instead of having multiple implementations or needing to redirect the output via an SLF4J bridging module.
Also consider the case of third-party libraries. I have dependencies in my application that are hard coded to log to about three different logging implementations. If they instead used SLF4J, the end user could choose whichever implementation they were already using in their app, instead of having multiple implementations or needing to redirect the output via an SLF4J bridging module.
slf4j was useful for me in a switch from log4j to logback.
Other useful tool for logging, that integrate with logback or log4j is sentry: https://github.com/getsentry/sentry
And to use it with java: https://github.com/kencochrane/raven-java
Other useful tool for logging, that integrate with logback or log4j is sentry: https://github.com/getsentry/sentry
And to use it with java: https://github.com/kencochrane/raven-java
Yes, we did, from log4j to logback. Due to using slf4j, no code changes were required.
I'm a fan of simplifying the server side, one I'd recommend for the latter part of the slideshow he referenced:
http://www.sparkjava.com/
http://www.sparkjava.com/
...or just use JAX-RS, which is simple, standardised, and has good implementations (such as RESTEasy and Jersey).
I suppose but SparkJava has as its core philosophy, build as a runnable jar from the start which is in line with what the article was discussing in its referential prohibition of conventional app server approaches.
edit: Further to the point of getting Java server technology to a simpler to understand, design, debug and maintain place, SparkJava makes it insanely easy to do REST services eliminating both Annotations and heavy configuration. I've also speed tested my side projects on SparkJava and they are unreal fast.
edit: Further to the point of getting Java server technology to a simpler to understand, design, debug and maintain place, SparkJava makes it insanely easy to do REST services eliminating both Annotations and heavy configuration. I've also speed tested my side projects on SparkJava and they are unreal fast.
cool article. I wish if someone can give a Java update like this once every year!
As far as packaging and deployment of a native executable goes, I thought it odd to not mention Excelsior JET (http://www.excelsiorjet.com) as I believe that's the only Java compiler that can handle all of the JDK (gcj isn't there yet).
Native executables are not always what you want. For example, they don't get security patches.
That's total nonsense. Binaries (or shared libraries they depend on) can be and are updated, on all sorts of systems. In many cases it's as simple as overwriting the existing binary with the updated version.
Heck, the runtimes you're advocating are generally installed as binaries. If they can get updated, then so can any other binary.
And if a user doesn't bother to update a given application's binary when a critical flaw of some sort is found, then there's a very good chance they wouldn't bother to update any runtimes that are installed, either. This is true even when some sort of update notification and installation process is offered. The end result is the same in either case: the update is not applied.
Runtimes shouldn't be portrayed as any better in this case, when they're generally no different than any other binary.
Heck, the runtimes you're advocating are generally installed as binaries. If they can get updated, then so can any other binary.
And if a user doesn't bother to update a given application's binary when a critical flaw of some sort is found, then there's a very good chance they wouldn't bother to update any runtimes that are installed, either. This is true even when some sort of update notification and installation process is offered. The end result is the same in either case: the update is not applied.
Runtimes shouldn't be portrayed as any better in this case, when they're generally no different than any other binary.
Anything can be done in many different ways. It all comes down to convenience. When a library is found to have a security flaw, you can either upgrade all binaries, or just upgrade one runtime. You could say that your OS is just a binary, too, and that's true. But it's a binary that adds a lot of convenience. A lot of people think runtimes add convenience, too. You obviously don't agree, and that's fine: use whatever works best for you.
http://xkcd.com/378/
http://xkcd.com/378/
gcj is dead since 2009.
The only alternatives, besides Excelsior, is the new RoboVM, Websphere Realtime JVM, Aicas, Aonix among a few other ones.
And if Graal eventually replaces Hotspot on the reference JVM, it might be that Truffle comes along as well.
The only alternatives, besides Excelsior, is the new RoboVM, Websphere Realtime JVM, Aicas, Aonix among a few other ones.
And if Graal eventually replaces Hotspot on the reference JVM, it might be that Truffle comes along as well.
As someone not very familiar with Java can someone expand on what he means by this: "Every library is usually packaged into its own JAR, and merging all dependencies into a single JAR, might cause collisions, especially with packaged resources (non-class files)."
When you merge multiple libraries into a single jar, it is possible that two libraries might have two (unpackaged) resources with the same name that will collide on the JVM's classpath. Or, perhaps more annoying, you might end up with an unexpected version of a (packaged) library defined class than you expected.
A common example is with logging config files (e.g., logback.xml). Some libraries ship with their own (or a library's dependency). It's not uncommon in a large project to wind up with several logback.xml files littered around from libraries that included their own. These are called "resources" because they aren't code but they are included with the compiled distributable and are placed on the classpath.
Of course, your project probably includes its own logback.xml in your resources directory.
Libraries are typically distributed as JAR files, so when you include their JAR on your classpath you're also including their logback.xml in that JAR.
Now, when you go to package up an uber-JAR for your project, whatever packaging tool you use has to merge all those files into one classpath. So now it's found collisions: there are multiple, different logback.xml files on your classpath but you can only have one. How does the tool pick?
So now you have to specify a "merge strategy" to make sure that only your logback.xml is merged in and the rest are discarded.
logback.xml is just one example. Typically Java classes themselves merge just fine because the packaging tool can dedup by fully-qualified class name which is almost never shared across projects. So even if two different dependencies have a Utils class, the FQCN is different. com.foo.bar.Utils is not the same as com.bar.baz.Utils, so there's no conflict.
Conflicts can also arise from libraries which include different versions of the same transitive dependency. Generally the fix here is to exclude a transitive dependency from one of your direct dependencies so as to only include one version of that library. A common culprit is Apache Commons since so many Java libraries rely on it.
Of course, your project probably includes its own logback.xml in your resources directory.
Libraries are typically distributed as JAR files, so when you include their JAR on your classpath you're also including their logback.xml in that JAR.
Now, when you go to package up an uber-JAR for your project, whatever packaging tool you use has to merge all those files into one classpath. So now it's found collisions: there are multiple, different logback.xml files on your classpath but you can only have one. How does the tool pick?
So now you have to specify a "merge strategy" to make sure that only your logback.xml is merged in and the rest are discarded.
logback.xml is just one example. Typically Java classes themselves merge just fine because the packaging tool can dedup by fully-qualified class name which is almost never shared across projects. So even if two different dependencies have a Utils class, the FQCN is different. com.foo.bar.Utils is not the same as com.bar.baz.Utils, so there's no conflict.
Conflicts can also arise from libraries which include different versions of the same transitive dependency. Generally the fix here is to exclude a transitive dependency from one of your direct dependencies so as to only include one version of that library. A common culprit is Apache Commons since so many Java libraries rely on it.
Speaking of Java logging, that's one area that drives me batty. The lack of a good standard Java logging library caused a multitude of logging libraries to be written. Different libraries then of course use different logging libraries, so each logging library generally has a way to wrap all the others, and they all work in different ways. Getting consistent and controllable logging (e.g. ability to turn on or off logging for a given thing) can sometimes be almost impossible to understand.
It's a sad story. The proliferation of logging libraries was the problem that Commons Logging aimed to solve by providing a facade over all of them. But because it screws up classloading (i forget the details, but it's serious), eventually a critical mass of people needed to move off it. java.util.logging tried to fix the problems and canonise a standard interface by putting in the JDK, but it got so many things wrong that it wasn't widely adopted. SLF4J finally came along as a very well-implemented facade that has gained wide adoption. It would say it is a de facto standard.
Only then JBoss decided that all their stuff (including Hibernate) needed a facade of its own, which manages to not quite properly interact with SLF4J!
It's almost enough to drive me to printlns.
Only then JBoss decided that all their stuff (including Hibernate) needed a facade of its own, which manages to not quite properly interact with SLF4J!
It's almost enough to drive me to printlns.
Agree, and will add that part of my frustration comes from the fact that nobody really cares about logging. You just want it to work.
OT, but I think this is my biggest complaint with Javascript. There's an explosion of invention and creativity with the language (because of its privileged position as the foundation of browser programming), but little standardization for the things I don't really care about.
OT, but I think this is my biggest complaint with Javascript. There's an explosion of invention and creativity with the language (because of its privileged position as the foundation of browser programming), but little standardization for the things I don't really care about.
Each JAR is just a zip file. It can be convenient to merge all those JAR files into one for deployment so you don't have to have all the JAR files lying around in the system and upgrades are just "replace this one JAR file" (among other reasons).
Since each JAR is just a zip if you have two files in seperate JARs with the same path you'll have a problem when you try to merge them.
Since each JAR is just a zip if you have two files in seperate JARs with the same path you'll have a problem when you try to merge them.
Class loaders work in a tree, and there typically isn't sharing between sibling branches. I believe that visibility is up the tree, and never across. So, you can end up with two different versions of a library existing quite happily in the same process space. Collapsing them down into a single [ejsw?]ar file results in the tree being collapsed down too.
All of a sudden all of the log4j.xml files that everyone has sprinkled everywhere end up overwriting each other...
All of a sudden all of the log4j.xml files that everyone has sprinkled everywhere end up overwriting each other...
FWIW, classloaders don't have to work as a tree. The JDK itself has a simple model with three classloaders in a hierarchy (bootstrap, extension, and system), which is a degenerate tree, and application servers have traditionally had a more general tree, but other topologies are possible. But OSGi and JBoss Modules have directed graphs of classloaders.
Very nice article. Capsule looks promising in regards to packaging. I'm going to have give 'er a try. One could also use the fatjar[1] or application[2] plugins (within the context of gradle).
[1] https://github.com/musketyr/gradle-fatjar-plugin [2] http://www.gradle.org/docs/current/userguide/application_plu...
[1] https://github.com/musketyr/gradle-fatjar-plugin [2] http://www.gradle.org/docs/current/userguide/application_plu...
This article does not mention OSGI or Jigsaw with one word but claims to guide to modern Java development.
Seems they never run any large scale EE projects yet :)
Ah, OSGI. OSGI is a bit of an enigma. People who use it think it's great, and, by the sound of it, assume everyone else is using it. People who don't use it think it's some weird thing from the turn of the century that nobody actually uses.
I do think a well-rounded Java developer should know about OSGI (i have to confess that i don't). But it would be mistaken to think that it's mainstream.
I do think a well-rounded Java developer should know about OSGI (i have to confess that i don't). But it would be mistaken to think that it's mainstream.
I've found OSGi to be an absolute nightmare. Classloader hacks is not the way to solve Java's dependency problem, it needs to be baked in the language IMO.
I think it's one of those ideas that sounds great in theory, but in reality it all falls apart.
Even if it worked as advertised, fact is many 3rd party libraries have all sorts of issues with OSGi. For example, my company is stuck on Jersey 1.x because 2.x doesn't work properly with OSGi.
Testing is much slower and harder to write. Testing seems to be an afterthought....
I think you can get most of the benefits of OSGi simply by using proper dependency management (Gradle). Use the Single Responsibility Principle and IoC and you'll get good modular code. These are much easer to do, have low risk, and are easy to test.
JMO
I think it's one of those ideas that sounds great in theory, but in reality it all falls apart.
Even if it worked as advertised, fact is many 3rd party libraries have all sorts of issues with OSGi. For example, my company is stuck on Jersey 1.x because 2.x doesn't work properly with OSGi.
Testing is much slower and harder to write. Testing seems to be an afterthought....
I think you can get most of the benefits of OSGi simply by using proper dependency management (Gradle). Use the Single Responsibility Principle and IoC and you'll get good modular code. These are much easer to do, have low risk, and are easy to test.
JMO
How does profiling Java applications work with JIT. Should you wait until the code has been optimized before profiling, or these tools smart enough to realize that the performance of code changes as the application runs?
I think as soon as any method takes up a noticeable part of execution time, it has been optimized all the way by the Hotspot or whatever your JVM uses.
After all, you want to profile your app's hot spots ;-)
After all, you want to profile your app's hot spots ;-)
The article mentions a tool called Flight Recorder for profiling, but it appears this is commercial and not available in the openjdk?