Job Forking

As of Obsidian 3.0.0, Obsidian can run each job in its own JVM. This allows for a number of possibilities, including support for dynamic changes to your deployed jobs (i.e. hot-swapping of JARs). In theory, you could even specialize classpaths on a per job basis by customizing the provided execution scripts.

To ensure backwards compatibility with older versions of Obsidian, this feature is disabled by default. Standalone deployments support forking out-of-the-box, but it is not active until you enable it as defined in Configuring Job Forking.

Note: In order to support hot-swapping of JARs that contain compiled jobs, Obsidian expects the defined parameter rules of the job to remain static. There is no provision to detect changes to these and support updating a job that previously wouldn't pass validation.

Configuring Job Forking

Enabling Job Forking - Cluster

First you must enable the functionality on the cluster. This is a System Parameter found in the Job Spawner category. You may enable it via the Admin UI, via one of the provided APIs (Embedded API or REST API) or the Initializing and Restoring support.

Enabling Job Forking - Node

Once you've done that, you'll need to enable it on each desired node. Standalone Schedulers are already thus enabled. For other nodes, you will need to set the appropriate properties value entry.

com.carfey.obsidian.jvmJobForkingEnabledOnThisNode=true

In a Standalone Scheduler, Job Forking will be functional on next start/restart without any other necessary changes. In other deployments or if you wish to further customize, additional changes will be required.

Script Location

You may wish to specify the forking script location for other deployments ( Embedded, Combined Scheduler and Admin Web Application), or for other reasons.

com.carfey.obsidian.forkedJobScriptLocation=/Obsidian-3.0.0

This property is the location of the fork scripts. Obsidian is bundled with obsidianForkedJob.bat and obsidianForkedJob.sh. These are the required script names. If you wish to customize these scripts or use your own, you are free to do so. But the script names must be as specified. Details about the Forked Obsidian Job Runner will be helpful in your customization efforts.

Classpath Override

You may wish to override the default classpath that is used by the forked execution instance. Or for deployment types other than Standalone Scheduler, you can use this property to specify the classpath.

com.carfey.obsidian.forkedJobscriptClasspathOverride=/home/user/workspace/obsidian/bin:/home/user/workspace/obsidian/lib/activation-1.1.jar:/home/user/workspace/obsidian/lib/mail-1.4.jar:/home/user/workspace/obsidian/lib/dom4j-1.6.1.jar:/home/user/workspace/obsidian/lib/obsidian.jar:/home/user/workspace/obsidian/lib/log4j-1.2.9.jar:/home/user/workspace/obsidian/lib/gson-2.2.2.jar:/home/user/workspace/obsidian/lib/bsh-2.0b4.jar:/home/user/workspace/obsidian/lib/groovy-all-2.1.8.jar:/home/user/workspace/obsidian/lib/jython-standalone-2.5.3.jar:/home/user/workspace/obsidian/lib/mariadb-java-client-1.1.5.jar

The sample demonstrates usage in an embedded Obsidian instance running inside an Eclipse project. The default classpath used in the obsidianForkedJob.sh/.bat script is built automatically assuming a Standalone Scheduler deployment. As such, it uses the jars in the standalone/ directory for building the classpath. If you require custom classpath on a per job basis, modification of the script(s) will be required.

How Does Job Forking Work?

Job Forking is implemented by the Obsidian runtime invoking a script for each job execution to be forked. This script in turn invokes a JVM calling an Obsidian class' main method providing the appropriate arguments to start the single-job execution Obsidian runtime.

Script Arguments

Obsidian calls the script with 15 arguments.

1. job_history_id - the job execution instance identifier
2. job_nickname
3. job_class
4. running_host
5. stack_file - used by the Scheduler Node to be aware of execution exceptions
6. db_url
7. db_jndi
8. db_user
9. db_pass
10. db_max_conn
11. db_conn_timeout
12. db_unused_conn_timeout
13. db_table_prefix
14. db_schema
15. classpath_override - optionally provided if the com.carfey.obsidian.forkedJobscriptClasspathOverride property is present.

Forked Obsidian Job Runner

Should you wish to write your own obsidianForkedJob.sh/obsidianForkedJob.bat or wish to modify the one(s) provided, you'll need to know how to invoke the Obsidian Forked Job Runtime. This Obsidian class is com.carfey.ops.job.ForkedJob. Its main method must be called with 2 arguments.

1. job_history_id
2. stack_file

A number of the other arguments are available to be passed as System Properties for db connectivity (if required) as noted below:

-Dcom.carfey.obsidian.db.url=$6 -Dcom.carfey.obsidian.db.jndiType=$7 -Dcom.carfey.obsidian.db.userId=$8 \ -Dcom.carfey.obsidian.db.password=$9 -Dcom.carfey.obsidian.db.maxConnections=${10} -Dcom.carfey.obsidian.db.connectionTimeout=${11} \ -Dcom.carfey.obsidian.db.unusedPoolConnectionTimeoutSeconds=${12} -Dcom.carfey.obsidian.db.tablePrefix=${13} -Dcom.carfey.obsidian.db.schema=${14}

Per Job Customizations - Classpaths and More

Three other arguments are provided as context for additional customization efforts, such as having alternate classpaths for particular jobs, classpath library ordering, controlling which java executable is used and so on. Any deterministic variation that might be required theoretically could be supported. Please contact the Obsidian Support team to discuss your needs or should you have any questions or concerns.

The script arguments job_nickname (3), job_class (4) and running_host (5) can be utilized within the script to determine what specialization to apply. This can be used in conjunction with the default classpath, the classpath override (15) or on its own.

For example, if you wanted to use a custom classpath if the job_class is of type com.carfey.ops.job.OldLibrariesJob and another one when the job nickname is New Open Document Format Job, you might modify the shell script like this:

if [ "New Open Document Format Job" == "${2}"] ; then
	cp=/Obsidian-3.0.0/newlibs/odf-4.0-beta.jar:/Obsidian-3.0.0/libs/... rest of classpath here
elif [ "com.carfey.ops.job.OldLibrariesJob" == "${3}"] ; then
	cp=/Obsidian-3.0.0/oldlibs/my-old-lib-1.0a.jar:/Obsidian-3.0.0/libs/... rest of classpath here
elif [ "" != "${15}" ]  ; then
	cp=${15}
elif [ "$(expr substr $(uname -s) 1 6)" == "CYGWIN" ] ; then
	cp=$(find standalone -name "*.jar" -exec printf "{};" ';')
else
	cp=$(find standalone -name "*.jar" -exec printf :{} ';')
fi