Job Forking

As of Obsidian 3.0.0, Obsidian can run each job in its own JVM. This allows for a number of possibilities, including support for dynamic changes to your deployed jobs (i.e. hot-swapping of JARs). In theory, you could even specialize classpaths on a per job basis by customizing the provided execution scripts.

JVM job forking is disabled by default. Standalone deployments support forking out-of-the-box, but it is not active until you enable it as defined in Configuring Job Forking.

Note: In order to support hot-swapping of JARs that contain compiled jobs, Obsidian expects the defined parameter rules of the job to remain static. There is no provision to detect changes to these and support updating a job that previously wouldn't pass validation.

How Does Job Forking Work?

Job forking is implemented by the Obsidian runtime invoking a script for each job execution to be forked. This script in turn starts a new JVM which invokes an Obsidian class with the appropriate arguments to start the single-job execution. Forking can be customized

Configuring Job Forking

Enabling Job Forking - Cluster

To use forking, first you must enable a setting that will apply across the cluster. This is a system parameter found in the Job Spawner category. You may enable it through the admin web application, the Embedded API, REST API, or Obsidian's initialization & restore support.

Enabling Job Forking - Node

Once you've done that, you'll need to enable it on each desired node. Standalone scheduler instances are configured to support forking by default. For other nodes, you will need to set the appropriate property value.

com.carfey.obsidian.jvmJobForkingEnabledOnThisNode=true

In a standalone scheduler, job forking will be then be functional on next restart. In other deployments ,or if you wish to make additional customizations, additional changes will be required.

Script Location

Job forking uses a script for each job execution to start up a fresh JVM.

You may wish to specify the forking script location for other deployments, such as embedded schedulers and combined scheduler and admin web application instances, or for other reasons.

To override the script location, set the appropriate property value in your properties file:

com.carfey.obsidian.forkedJobScriptLocation=/Obsidian-3.0.0

Obsidian is bundled with obsidianForkedJob.bat and obsidianForkedJob.sh. These are the required script names. If you wish to customize these scripts or use your own, you are free to do so, but the script names cannot be changed. Review the notes on the forked job runner to assist your customization efforts.

Classpath Override

You may wish to override the default classpath that is used by the forked execution instance. Or for deployment types other than standalone schedulers, you can use this property to specify the classpath.

com.carfey.obsidian.forkedJobscriptClasspathOverride=/home/user/workspace/obsidian/bin:/home/user/workspace/obsidian/lib/activation-1.1.jar\
:/home/user/workspace/obsidian/lib/mail-1.4.jar:/home/user/workspace/obsidian/lib/dom4j-1.6.1.jar:/home/user/workspace/obsidian/lib/obsidian.jar\
:/home/user/workspace/obsidian/lib/log4j-1.2.9.jar:/home/user/workspace/obsidian/lib/gson-2.2.2.jar:/home/user/workspace/obsidian/lib/bsh-2.0b4.jar\
:/home/user/workspace/obsidian/lib/groovy-all-2.1.8.jar:/home/user/workspace/obsidian/lib/jython-standalone-2.5.3.jar\
:/home/user/workspace/obsidian/lib/mariadb-java-client-1.1.5.jar

The sample demonstrates usage in an embedded Obsidian instance running inside an Eclipse project. The default classpath used in the obsidianForkedJob.(sh|bat) script is built automatically assuming a standalone scheduler deployment. As such, it uses the jars in the standalone/ directory for building the classpath. If you require a custom classpath on a per job basis, modification of the forking script(s) will be required.

Customization

Script Arguments

Obsidian calls the script with 15 arguments.

1. job_history_id - the job execution instance identifier
2. job_nickname
3. job_class
4. running_host
5. stack_file - used by the Scheduler Node to be aware of execution exceptions
6. db_url
7. db_jndi
8. db_user
9. db_pass
10. db_max_conn
11. db_conn_timeout
12. db_unused_conn_timeout
13. db_table_prefix
14. db_schema
15. classpath_override - optionally provided if the com.carfey.obsidian.forkedJobscriptClasspathOverride property is present.

Forked Obsidian Job Runner

Should you wish to write your own obsidianForkedJob.sh/obsidianForkedJob.bat or wish to modify the one(s) provided, you'll need to know how to invoke the forked job runtime. The runtime is accessed by invoking the Obsidian class com.carfey.ops.job.ForkedJob. Its main method must be called with 2 arguments.

1. job_history_id
2. stack_file

A number of the other arguments are available to be passed as JVM system properties for database connectivity as noted below:

-Dcom.carfey.obsidian.db.url=$6 -Dcom.carfey.obsidian.db.jndiType=$7 -Dcom.carfey.obsidian.db.userId=$8 \
-Dcom.carfey.obsidian.db.password=$9 -Dcom.carfey.obsidian.db.maxConnections=${10} -Dcom.carfey.obsidian.db.connectionTimeout=${11} \
-Dcom.carfey.obsidian.db.unusedPoolConnectionTimeoutSeconds=${12} -Dcom.carfey.obsidian.db.tablePrefix=${13} -Dcom.carfey.obsidian.db.schema=${14}

Per Job Customizations - Classpaths and More

Three other arguments are provided as context for additional customization efforts, such as having alternate classpaths for particular jobs, classpath library ordering, controlling which Java executable is used and so on. Please contact the Obsidian support team to discuss your needs or with any questions you have.

The script arguments job_nickname, job_class and running_host can be utilized within the script to determine what specialization to apply. This can be used in conjunction with the default classpath, the classpath_override or on its own.

For example, if you wanted to use a custom classpath if the job_class is of type "com.example.OldLibrariesJob" and another one when the job nickname is "New Open Document Format Job", you might modify the shell script like this:

if [ "New Open Document Format Job" == "${2}"] ; then
	cp=/Obsidian-3.0.0/newlibs/odf-4.0-beta.jar:/Obsidian-3.0.0/libs/... rest of classpath here
elif [ "com.example.OldLibrariesJob" == "${3}"] ; then
	cp=/Obsidian-3.0.0/oldlibs/my-old-lib-1.0a.jar:/Obsidian-3.0.0/libs/... rest of classpath here
elif [ "" != "${15}" ]  ; then
	cp=${15}
elif [ "$(expr substr $(uname -s) 1 6)" == "CYGWIN" ] ; then
	cp=$(find standalone -name "*.jar" -exec printf "{};" ';')
else
	cp=$(find standalone -name "*.jar" -exec printf :{} ';')
fi