Granite - Offloading - Day

Granite - OffloadingMarc Pfaff | Basel© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Why Offloading§ Scalability in AEM:§§DAM IngestionNon-Clustered installation requirement§ The term Offloading:§§In AEM used for all things job distribution and topology in clustered and nonclusteredinstallations, e.g. ‘Offloading Browser’More technically it’s ‘only’ a little add-on in Granite to Sling Job Distribution forhandling non-clustered installations© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.2

Overview – Where to fit in the stackAEMDAM IngestionGraniteOffloadingWorkflow DistributionSlingDiscoveryJob Distribution© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.3

Overview – Where to fit in the stackAEMDAM IngestionGraniteOffloadingWorkflow DistributionSlingDiscoveryJob Distribution© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.4

Sling Job DistributionTopologyInstanceSling ID: 1InstanceSling ID: 2InstanceSling ID: 3InstanceSling ID: 4Job ConsumerTopic: AJob ConsumerTopic: BJob ConsumerTopic: CJob ManagerJob ManagerJob ManagerJob Manager© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.5

Sling Job DistributionTopologyInstanceSling ID: 1InstanceSling ID: 2InstanceSling ID: 3InstanceSling ID: 4JobJob ConsumerTopic: AJob ConsumerTopic: BJob ConsumerTopic: CAJob ManagerJob ManagerJob ManagerJob ManagerA:2© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.6

Sling Job DistributionTopologyInstanceSling ID: 1InstanceSling ID: 2InstanceSling ID: 3InstanceSling ID: 4JobJob ConsumerTopic: AJob ConsumerTopic: BJob ConsumerTopic: CBJob ManagerJob ManagerJob ManagerJob ManagerB:3© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.7

Sling Job DistributionTopologyInstanceSling ID: 1InstanceSling ID: 2InstanceSling ID: 3InstanceSling ID: 4JobJob ConsumerTopic: AJob ConsumerTopic: BJob ConsumerTopic: CCJob ManagerJob ManagerJob ManagerJob ManagerC:4?© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.8

Sling Job DistributionTopologyInstanceSling ID: 1InstanceSling ID: 2InstanceSling ID: 3InstanceSling ID: 4JobJob ConsumerTopic: AJob ConsumerTopic: BJob ConsumerTopic: CCJob ManagerJob ManagerJob ManagerJob ManagerC:4OffloadingFramework© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.9

Offloading Framework - OverviewTopology§ Detects offloading jobs§ Transport of job and jobpayload between originand target instanceJobInstanceSling ID: 1Job ConsumerTopic: AInstanceSling ID: 2Job ConsumerTopic: C§ Uses replication for thetransportC1Job ManagerReplicationJob Manager§ No distribution of jobs§ No execution of jobs2OffloadingJob Manager4OffloadingJob Manager63 5C:2C:2© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.10

Offloading Framework – Replication§ Replication agents arecreated automatically§ Uses namingconvention§ Needs manualadjustments(replication user)AuthorSling ID: 1offloading_2(outgoing agent)offloading_reverse_2(reverse agent)WorkerSling ID: 2offloading_outbox(outbox agent)© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.11

Offloading Framework – Payload§ Defines the content to be transported between the instances§ Properties on the job payload§ Takes comma separated list of paths§ Used to build the replication package§ The job itself is implicitly added by the offloading framework§ Offloading job input§Property: OffloadingJobProperties. INPUT_PAYLOAD(offloading.input.payload)§ Offloading job output§Property: OffloadingJobProperties. OUTPUT_PAYLOAD(offloading.output.payload)© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.12

Offloading Browser - UI§ Configures Job Consumers§§Configures the topic white/black listing properties of each instanceWhat jobs to execute on what instance§ Configures the distribution§ Configuration applies for both, clustered and non-clustered installations© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.13

Offloading Browser - UI© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.14

Offloading Browser - UI© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.15

Offloading Browser - UI© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.16

Overview – Where to fit in the stackAEMDAM IngestionGraniteOffloadingWorkflow DistributionSlingDiscoveryJob Distribution© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.17

Workflow Distribution§ New JobConsumer§§§§§Class: WorkflowOffloadingJobConsumerTopic: com/adobe/granite/workflow/offloadingCan launch new workflowsExpects the workflow model on the job payloadExpects the workflow payload on the job payload§ For use with clustered and non-clustered installations© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.18

Workflow Distribution – Job Consumer (Simplified)!@Component!@Service!@Properties({!@Property(name = JobConsumer.PROPERTY_TOPICS, value = WorkflowOffloadingJobConsumer.TOPIC)!})!public class WorkflowOffloadingJobConsumer implements JobConsumer {!!// the topic for use for this job consumer ! ! !!public static final String TOPIC = "com/adobe/granite/workflow/offloading";!!// the job payload properties!public static final String WORKFLOW_OFFLOADING_MODEL = "offloading.workflow.model";!public static final String WORKFLOW_OFFLOADING_PAYLOAD = "offloading.workflow.payload”;!!public JobResult process(Job job) {!// read workflow model and payload from job payload!String modelPath= job.getProperty(WORKFLOW_OFFLOADING_MODEL , "");!String payloadPath= job.getProperty(WORKFLOW_OFFLOADING_PAYLOAD , "");!!// get/create WorkflowSession, WorkflowModel and WorkflowData objects!WorkflowSession wfSession = ..;!WorkflowModel wfModel = ..;!WorkflowData wfData = ..;!!// start the workflow!wfSession.startWorkflow(wfModel, wfData, metaData);!!// all good!return JobResutl.OK;!}!}!© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.19

Workflow Distribution – Job Consumer (Simplified)!@Component!@Service!@Properties({!@Property(name = JobConsumer.PROPERTY_TOPICS, value = WorkflowOffloadingJobConsumer.TOPIC)!})!public class WorkflowOffloadingJobConsumer implements JobConsumer {!!// the topic for use for this job consumer ! ! !!public static final String TOPIC = "com/adobe/granite/workflow/offloading";!!// the job payload properties!public static final String WORKFLOW_OFFLOADING_MODEL = "offloading.workflow.model";!public static final String WORKFLOW_OFFLOADING_PAYLOAD = "offloading.workflow.payload”;!!!!public JobResult process(Job job) {!// read workflow model and payload from job payload!String modelPath= job.getProperty(WORKFLOW_OFFLOADING_MODEL , "");!String payloadPath= job.getProperty(WORKFLOW_OFFLOADING_PAYLOAD , "");!// get/create WorkflowSession, WorkflowModel and WorkflowData objects!WorkflowSession wfSession = ..;!WorkflowModel wfModel = ..;!WorkflowData wfData = ..;!// start the workflow!wfSession.startWorkflow(wfModel, wfData, metaData);!!// all good!return JobResutl.OK;!}!}!• Create service component• Must register with topic• Implement new JobConsumerinterface© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.20

Workflow Distribution – Job Consumer (Simplified)!@Component!@Service!@Properties({!@Property(name = JobConsumer.PROPERTY_TOPICS, value = WorkflowOffloadingJobConsumer.TOPIC)!})!public class WorkflowOffloadingJobConsumer implements JobConsumer {!!// the topic for use for this job consumer ! ! !!public static final String TOPIC = "com/adobe/granite/workflow/offloading";!!// the job payload properties!public static final String WORKFLOW_OFFLOADING_MODEL = "offloading.workflow.model";!public static final String WORKFLOW_OFFLOADING_PAYLOAD = "offloading.workflow.payload”;!!public JobResult process(Job job) {!• Define the job topic// read workflow model and payload from job payload!String modelPath= job.getProperty(WORKFLOW_OFFLOADING_MODEL , "");!String payloadPath= job.getProperty(WORKFLOW_OFFLOADING_PAYLOAD , "");!!!// get/create WorkflowSession, WorkflowModel and WorkflowData objects!WorkflowSession wfSession = ..;!WorkflowModel wfModel = ..;!WorkflowData wfData = ..;!// start the workflow!wfSession.startWorkflow(wfModel, wfData, metaData);!!// all good!return JobResutl.OK;!}!}!© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.21

Workflow Distribution – Job Consumer (Simplified)!@Component!@Service!@Properties({!@Property(name = JobConsumer.PROPERTY_TOPICS, value = WorkflowOffloadingJobConsumer.TOPIC)!})!public class WorkflowOffloadingJobConsumer implements JobConsumer {!!// the topic for use for this job consumer ! ! !!public static final String TOPIC = "com/adobe/granite/workflow/offloading";!!// the job payload properties!public static final String WORKFLOW_OFFLOADING_MODEL = "offloading.workflow.model";!public static final String WORKFLOW_OFFLOADING_PAYLOAD = "offloading.workflow.payload”;!!public JobResult process(Job job) {!// read workflow model and payload from job payload!String modelPath= job.getProperty(WORKFLOW_OFFLOADING_MODEL , "");!String payloadPath= job.getProperty(WORKFLOW_OFFLOADING_PAYLOAD , "");!!// get/create WorkflowSession, WorkflowModel and WorkflowData objects!WorkflowSession wfSession = ..;!WorkflowModel wfModel = ..;!WorkflowData wfData = ..;!!// start the workflow!wfSession.startWorkflow(wfModel, wfData, metaData);!!// all good!return JobResutl.OK;!}!}!• Access job properties (payload)• Read workflow model and payloadfrom job properties© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.22

Workflow Distribution – Job Consumer (Simplified)!@Component!@Service!@Properties({!@Property(name = JobConsumer.PROPERTY_TOPICS, value = WorkflowOffloadingJobConsumer.TOPIC)!})!public class WorkflowOffloadingJobConsumer implements JobConsumer {!!// the topic for use for this job consumer ! ! !!public static final String TOPIC = "com/adobe/granite/workflow/offloading";!!// the job payload properties!• Workflow specificpublic static final String WORKFLOW_OFFLOADING_MODEL = "offloading.workflow.model";!public static final String WORKFLOW_OFFLOADING_PAYLOAD = "offloading.workflow.payload”;!!!!public JobResult process(Job job) {!// read workflow model and payload from job payload!String modelPath= job.getProperty(WORKFLOW_OFFLOADING_MODEL , "");!String payloadPath= job.getProperty(WORKFLOW_OFFLOADING_PAYLOAD , "");!// get/create WorkflowSession, WorkflowModel and WorkflowData objects!WorkflowSession wfSession = ..;!WorkflowModel wfModel = ..;!WorkflowData wfData = ..;!// start the workflow!wfSession.startWorkflow(wfModel, wfData, metaData);!!// all good!return JobResutl.OK;!}!}!• Use workflow API to start workflowfor the given model and payload© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.23

Workflow Distribution – Job Consumer (Simplified)!@Component!@Service!@Properties({!@Property(name = JobConsumer.PROPERTY_TOPICS, value = WorkflowOffloadingJobConsumer.TOPIC)!})!public class WorkflowOffloadingJobConsumer implements JobConsumer {!!// the topic for use for this job consumer ! ! !!public static final String TOPIC = "com/adobe/granite/workflow/offloading";!!// the job payload properties!public static final String WORKFLOW_OFFLOADING_MODEL = "offloading.workflow.model";!public static final String WORKFLOW_OFFLOADING_PAYLOAD = "offloading.workflow.payload”;!!public JobResult process(Job job) {!// read workflow model and payload from job payload!String modelPath= job.getProperty(WORKFLOW_OFFLOADING_MODEL , "");!String payloadPath= job.getProperty(WORKFLOW_OFFLOADING_PAYLOAD , "");!!// get/create WorkflowSession, WorkflowModel and WorkflowData report objects! back the job statusWorkflowSession wfSession = ..;!WorkflowModel wfModel = ..;!WorkflowData wfData = ..;!!// start the workflow!wfSession.startWorkflow(wfModel, wfData, metaData);!!// all good!return JobResutl.OK;!}!}!• Use JobResult enumeration to© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.24

Overview – Where to fit in the stackAEMDAM IngestionGraniteOffloadingWorkflow DistributionSlingDiscoveryJob Distribution© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.25

DAM Ingestion§ Default ingestion workflow: “DAM Update Asset”§Load is put on the instance where the workflow is started, usually the author§ New ingestion workflow: “DAM Update Asset Offloading”§§§§§§Needs to be manually enabled by changing the workflow launcherNew workflow model with a single step: AssetOffloadingProcessUses WorkflowExternalProcess APICreates a new job on topic: com/adobe/granite/workflow/offloadingAllows distributing the default ingestion workflowLoad is put on the instance where the job is distributed to§ Can be used to distribute in clustered and non-clustered installations© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.26

DAM Ingestion – Create Job (from workflow step)!@Component!@Service!public class AssetOffloadingProcess implements WorkflowExternalProcess {!@Reference!private JobManager jobManager;!!private static final String TOPIC = "com/adobe/granite/workflow/offloading";!!public Serializable execute(WorkItem workItem, WorkflowSession workflowSession, MetaDataMap metaDataMap){!Asset asset = ..; !String workflowModel = “/etc/workflow/models/dam/update_asset/jcr:content/model”; !String workflowPayload = “/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg”;!!ValueMap jobProperties = new ValueMapDecorator(new HashMap());!jobProperties.put(WORKFLOW_OFFLOADING_MODEL, workflowModel);!jobProperties.put(WORKFLOW_OFFLOADING_PAYLOAD, workflowPayload);!!String offloadingInput = “/etc/workflow/models/dam/update_asset/jcr:content/model, !/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” ;!String offloadingOutput = “/etc/workflow/models/dam/update_asset/jcr:content/model, !/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” ;!!jobProperties.put(OffloadingJobProperties.INPUT_PAYLOAD.propertyName(), offloadingInput);!jobProperties.put(OffloadingJobProperties.OUTPUT_PAYLOAD.propertyName(), offloadingOutput);!!Job offloadingJob = jobManager.addJob(TOPIC, null, jobProperties);!return offloadingJob.getId();!}!!public boolean hasFinished(Serializable externalProcessId, ..){!// returns null, if job is finished !Job offloadingJob = jobManager.getJobById((String) externalProcessId);!return offloadingJob == null;!}!}!!© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.27

DAM Ingestion – Create Job (from workflow step)!@Component!@Service!public class AssetOffloadingProcess implements WorkflowExternalProcess {!@Reference!private JobManager jobManager;!!private static final String TOPIC = "com/adobe/granite/workflow/offloading";!!public Serializable execute(WorkItem workItem, WorkflowSession workflowSession, MetaDataMap metaDataMap){!Asset asset = ..; !String workflowModel = “/etc/workflow/models/dam/update_asset/jcr:content/model”; !String workflowPayload = “/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg”;!!• Create service componentValueMap jobProperties = new ValueMapDecorator(new HashMap());!jobProperties.put(WORKFLOW_OFFLOADING_MODEL, workflowModel);!jobProperties.put(WORKFLOW_OFFLOADING_PAYLOAD, workflowPayload);!!• Reference JobManager serviceString offloadingInput = “/etc/workflow/models/dam/update_asset/jcr:content/model, !/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” ;!String offloadingOutput = “/etc/workflow/models/dam/update_asset/jcr:content/model, !/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” ;!!jobProperties.put(OffloadingJobProperties.INPUT_PAYLOAD.propertyName(), offloadingInput);!jobProperties.put(OffloadingJobProperties.OUTPUT_PAYLOAD.propertyName(), offloadingOutput);!!Job offloadingJob = jobManager.addJob(TOPIC, null, jobProperties);!return offloadingJob.getId();!}!!public boolean hasFinished(Serializable externalProcessId, ..){!// returns null, if job is finished !Job offloadingJob = jobManager.getJobById((String) externalProcessId);!return offloadingJob == null;!}!}!!WorkflowExternalProcess interface© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.28

DAM Ingestion – Create Job (from workflow step)!@Component!@Service!public class AssetOffloadingProcess implements WorkflowExternalProcess {!@Reference!private JobManager jobManager;!!private static final String TOPIC = "com/adobe/granite/workflow/offloading";!!public Serializable execute(WorkItem workItem, WorkflowSession workflowSession, MetaDataMap metaDataMap){!Asset asset = ..; !String workflowModel = “/etc/workflow/models/dam/update_asset/jcr:content/model”; !String workflowPayload = “/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg”;!!ValueMap jobProperties = new ValueMapDecorator(new HashMap());!jobProperties.put(WORKFLOW_OFFLOADING_MODEL, workflowModel);!jobProperties.put(WORKFLOW_OFFLOADING_PAYLOAD, workflowPayload);!!String offloadingInput = “/etc/workflow/models/dam/update_asset/jcr:content/model, !/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” • DAM and Workflow ;! specificString offloadingOutput = “/etc/workflow/models/dam/update_asset/jcr:content/model,• Resolve to Asset!/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” ;!!• Read model from meta datajobProperties.put(OffloadingJobProperties.INPUT_PAYLOAD.propertyName(), offloadingInput);!jobProperties.put(OffloadingJobProperties.OUTPUT_PAYLOAD.propertyName(), offloadingOutput);!path!Job offloadingJob = jobManager.addJob(TOPIC, null, jobProperties);!return offloadingJob.getId();!}!!public boolean hasFinished(Serializable externalProcessId, ..){!// returns null, if job is finished !Job offloadingJob = jobManager.getJobById((String) externalProcessId);!return offloadingJob == null;!}!}!!• Read workflow payload from Asset© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.29

DAM Ingestion – Create Job (from workflow step)!@Component!@Service!public class AssetOffloadingProcess implements WorkflowExternalProcess {!@Reference!private JobManager jobManager;!!private static final String TOPIC = "com/adobe/granite/workflow/offloading";!!public Serializable execute(WorkItem workItem, WorkflowSession workflowSession, MetaDataMap metaDataMap){!Asset asset = ..; !String workflowModel = “/etc/workflow/models/dam/update_asset/jcr:content/model”; !String workflowPayload = “/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg”;!!ValueMap jobProperties = new ValueMapDecorator(new HashMap());!jobProperties.put(WORKFLOW_OFFLOADING_MODEL, workflowModel);!jobProperties.put(WORKFLOW_OFFLOADING_PAYLOAD, workflowPayload);!!String offloadingInput = “/etc/workflow/models/dam/update_asset/jcr:content/model, !/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” ;!String offloadingOutput = “/etc/workflow/models/dam/update_asset/jcr:content/model, !/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” ;!!jobProperties.put(OffloadingJobProperties.INPUT_PAYLOAD.propertyName(), offloadingInput);!jobProperties.put(OffloadingJobProperties.OUTPUT_PAYLOAD.propertyName(), • ValueMap offloadingOutput);!for job properties!Job offloadingJob = jobManager.addJob(TOPIC, null, jobProperties);!return offloadingJob.getId();!}!!public boolean hasFinished(Serializable externalProcessId, ..){!// returns null, if job is finished !Job offloadingJob = jobManager.getJobById((String) externalProcessId);!return offloadingJob == null;!}!}!!• Put model and payload on jobproperties• Used by the JobConsumer© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.30

DAM Ingestion – Create Job (from workflow step)!@Component!@Service!public class AssetOffloadingProcess implements WorkflowExternalProcess {!@Reference!private JobManager jobManager;!!private static final String TOPIC = "com/adobe/granite/workflow/offloading";!!public Serializable execute(WorkItem workItem, WorkflowSession workflowSession, MetaDataMap metaDataMap){!• Comma separated list of pathsAsset asset = ..; !String workflowModel = “/etc/workflow/models/dam/update_asset/jcr:content/model”; !String workflowPayload = “/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg”;!• Only used for non-clustered!ValueMap jobProperties = new ValueMapDecorator(new HashMap());!jobProperties.put(WORKFLOW_OFFLOADING_MODEL, workflowModel);!jobProperties.put(WORKFLOW_OFFLOADING_PAYLOAD, workflowPayload);!!String offloadingInput = “/etc/workflow/models/dam/update_asset/jcr:content/model, !/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” ;!String offloadingOutput = “/etc/workflow/models/dam/update_asset/jcr:content/model, !/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” ;!!jobProperties.put(OffloadingJobProperties.INPUT_PAYLOAD.propertyName(), offloadingInput);!jobProperties.put(OffloadingJobProperties.OUTPUT_PAYLOAD.propertyName(), offloadingOutput);!!Job offloadingJob = jobManager.addJob(TOPIC, null, jobProperties);!return offloadingJob.getId();!}!!public boolean hasFinished(Serializable externalProcessId, ..){!// returns null, if job is finished !Job offloadingJob = jobManager.getJobById((String) externalProcessId);!return offloadingJob == null;!}!}!!• Build offloading payload properties• Put them on the job payload as well© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.31

DAM Ingestion – Create Job (from workflow step)!@Component!@Service!public class AssetOffloadingProcess implements WorkflowExternalProcess {!@Reference!private JobManager jobManager;!!private static final String TOPIC = "com/adobe/granite/workflow/offloading";!!public Serializable execute(WorkItem workItem, WorkflowSession workflowSession, MetaDataMap metaDataMap){!Asset asset = ..; !String workflowModel = “/etc/workflow/models/dam/update_asset/jcr:content/model”; !String workflowPayload = “/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg”;!!ValueMap jobProperties = new ValueMapDecorator(new HashMap());!• Create job using JobManagerjobProperties.put(WORKFLOW_OFFLOADING_MODEL, workflowModel);!jobProperties.put(WORKFLOW_OFFLOADING_PAYLOAD, workflowPayload);! service!• Use topic from job consumerString offloadingInput = “/etc/workflow/models/dam/update_asset/jcr:content/model, !/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg”• Put job payload;!propertiesString offloadingOutput = “/etc/workflow/models/dam/update_asset/jcr:content/model, • Return the jobId ! as the workflow/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” ;!process id (workflow specific)!jobProperties.put(OffloadingJobProperties.INPUT_PAYLOAD.propertyName(), offloadingInput);!jobProperties.put(OffloadingJobProperties.OUTPUT_PAYLOAD.propertyName(), offloadingOutput);!!Job offloadingJob = jobManager.addJob(TOPIC, null, jobProperties);!return offloadingJob.getId();!}!!public boolean hasFinished(Serializable externalProcessId, ..){!// returns null, if job is finished !Job offloadingJob = jobManager.getJobById((String) externalProcessId);!return offloadingJob == null;!}!}!!© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.32

DAM Ingestion – Create Job (from workflow step)!@Component!@Service!public class AssetOffloadingProcess implements WorkflowExternalProcess {!@Reference!private JobManager jobManager;!!private static final String TOPIC = "com/adobe/granite/workflow/offloading";!!public Serializable execute(WorkItem workItem, WorkflowSession workflowSession, MetaDataMap metaDataMap){!Asset asset = ..; !String workflowModel = “/etc/workflow/models/dam/update_asset/jcr:content/model”; !String workflowPayload = “/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg”;!!ValueMap jobProperties = new ValueMapDecorator(new HashMap());!jobProperties.put(WORKFLOW_OFFLOADING_MODEL, workflowModel);!jobProperties.put(WORKFLOW_OFFLOADING_PAYLOAD, workflowPayload);!!String offloadingInput = “/etc/workflow/models/dam/update_asset/jcr:content/model, !/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” • Workflow API ;! specific callbackString offloadingOutput = “/etc/workflow/models/dam/update_asset/jcr:content/model,• Process id = jobId,!from execute()/content/dam/geometrixx-outdoors/articles/downhill-ski-conditioning.jpg” ;!!• Query job by jobIdjobProperties.put(OffloadingJobProperties.INPUT_PAYLOAD.propertyName(), • Workflow offloadingInput);!step finished when job isjobProperties.put(OffloadingJobProperties.OUTPUT_PAYLOAD.propertyName(), offloadingOutput);!finished!Job offloadingJob = jobManager.addJob(TOPIC, null, jobProperties);!return offloadingJob.getId();!}!!public boolean hasFinished(Serializable externalProcessId, ..){!// returns null, if job is finished !Job offloadingJob = jobManager.getJobById((String) externalProcessId);!return offloadingJob == null;!}!}!!© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.33

Development - Recipe1. Choose a job topic2. Create JobConsumer component and register with topic chosen3. To create a new job use new JobManager.addJob() API with thetopic chosen and the job payload4. Add offloading payload to job payload5. Bundle and deploy JobConsumer on topology instances6. Enable/Disable the new topic on the instances, using OffloadingBrowser© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.34

Take awayAEMDAM IngestionGraniteOffloadingWorkflow DistributionSlingDiscoveryJob Distribution© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.35

Take awayAEMorg.apache.sling.discovery.apiDAM IngestionGranite• Build Topology• Cross-Cluster detection• Foundation for job distributionOffloadingWorkflow DistributionSlingDiscoveryJob Distribution© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.36

Take awayorg.apache.sling.eventAEMGranite• Uses Sling Discovery• New JobConsumer API and job topics• New JobManager API for creating new distributed jobs• Distributes jobsDAMbasedIngestionavailable job topics and job queueconfiguration.• Distributes in the whole topology, including clustered and nonclusteredinstances• Can execute cluster local jobs onlyOffloadingWorkflow DistributionSlingDiscoveryJob Distribution© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.37

Take awayAEMcom.adobe.granite.offloading.core• Builds on top of Sling Distributed Jobs• Does not perform distribution• Detects jobs distributed to non-clustered instances• Transports the jobs and payload to non-clusteredinstances• Uses replication for transport• Does not execute jobsDAM IngestionGraniteOffloadingWorkflow DistributionSlingDiscoveryJob Distribution© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.38

Take awaycom.adobe.granite.workflow.coreAEM• Defines a job consumer to distribute the execution ofwhole workflows• Defines topic com/adobe/granite/workflow/offloading• Implements WorkflowOffloadingJobConsumercomponent• Supports clustered and non clustered installationsDAM IngestionGraniteOffloadingWorkflow DistributionSlingDiscoveryJob Distribution© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.39

Take awayAEMDAM IngestionGraniteSlingOffloadingDiscoverycq-dam-coreWorkflow Distribution• Makes use of com/adobe/granite/workflow/offloadingtopic from Workflow Distribution• New workflow step (external step) that creates a newjob on topic com/adobe/granite/workflow/offloading• New “DAM Update Asset Job Offloading” Distribution workflow• Supports clustered and non clustered configurations© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.40

© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.