Using Job Trace to troubleshoot print job problems

Like every good story, a job trace has a beginning, a middle, and an end. It shows the journey of a print job from the user (origin node), through the other edge nodes chosen to replicate the job (replication nodes), and finally, its route to the printer (print candidate nodes). Calls between the edge nodes and the PaperCut Hive Cloud are visible to allow for troubleshooting.

To cater for more complex network environments, we have amended our best practices for super nodes guide to help you identify whether super nodes with local queues will help when the Edge Mesh encounters difficulties due to network segmentation, port, and protocol controls, or device capabilities.

Viewing the Job Trace for a print job

Job Trace is a diagnostics tool that PaperCut Support uses to help troubleshoot issues with print jobs in PaperCut Hive and Pocket. Job Trace basically allows an admin to trace a print job’s path from the time a print job is submitted to when it’s released at the printer.

Requirement - you must enable Diagnostic mode to access Job Trace.

1. In the Explore section, click Job Log.
   
   
2. In the Job Log list, click the required job to view the job details.
3. In the Details tab, scroll down to the bottom of the job log. Click the Diagnostics button, then click View job trace.
   
   

Here’s an example of the Header and Information at the start of a Job Trace:

The 4 sections of a Job Trace

The 4 sections of a Job Trace are:

• Header
• Information
• Highlighted issues
• Job Trace details 

Header

At the very top is the header, which contains the document name. Under that is the Job ID in the format of orgID/job-ID. PaperCut Support often asks for Job IDs and this is one place where you can find them. 

Information

This section provides a quick summary of the print job’s details, like printing edge candidates (a list of edge nodes that can print the job), printer name, last status, and printing edge node (the edge node that actually printed the job).

Highlighted issues

This section, as the name suggests, highlights any issues/errors from the Job Trace during the print job workflow. Click on each issue to view the details. If you’re lucky, the number of highlighted issues will be at [0].

Job Trace details

Finally, we have the actual Job Trace section. It details the workflow from when the print job was submitted to the Edge Mesh after the user clicked Print on the PaperCut Printer. 

The 3 major major printing functions that are documented in the Job Trace are the submission, replication, and release functions.

Submission 

The Job trace shows the following:

• The job’s submission status, either submitting or submitted.
• The job actions (Job Rules Color, 2 Sided, etc.).
• The edge node job analysis.

Replication 

The Job Trace shows the following:

• If cloud node is Off, it shows the other 2 edge nodes selected for replication. 
• If cloud node is On, it shows replication to the cloud node. 
• The first edge node’s document link (URI) that was used to copy the print job.

Release

The Job Trace shows the following:

• The edge node assigned to print the job. 
• The job delivery mode: IPP/IPPS mode or queue mode. 

Interpreting messages

Below are some of the errors we might see in the Job Trace.

documentcheck.inaccessible

When an edge node performs a URI check, the job log shows if the check was successful or failed. A failure could be caused by:

• users are on separate subnets
• the job wasn’t replicated
• the user’s device was offline at the time of replication.

printercheck.unreachable

Similar to documentcheck, a printercheck attempts to reach a printer via IPP/IPPS and when the printer is unreachable, this message is logged.

To prevent this error, we suggest using queues where possible and making sure that the printing edge node or super node can reach the printer(s) on the network.

jobrouter.checkprinting.timeout 

This timeout happens when the edge node that picks up a job for printing doesn’t call in to the PaperCut Cloud for 3 minutes. It’s assumed the edge node has stopped working for some reason in the middle of trying to print.

Edge nodes are meant to call in the progress of a job every minute while a job is in the “printing” state. If no progress is reported for 3 minutes, it’s concluded that the edge node has stopped responding and the job failed.

routing tasks were not created for job(s), skipping call to statemachine, and failed to process release request: failed to release few jobs, err <nil>

Workflow

A user prints a job to the PaperCut Printer. They walk over to the copier to release their print job but the job doesn’t appear in the print release list. The PaperCut Hive or Pocket admin console shows the job is ‘stuck’ in analyzing and can’t be canceled. 

routing tasks were not created for job(s), skipping call to statemachine

This error may indicate a problematic edge node failed to analyze the print job and the job failed to move beyond the analysis task. When the user goes to release their job at the copier, the job doesn’t appear in the release list. 

Informational-style messages

Failed to update job state: job state transition not allowed

Looks like an error, sounds like an error but this one isn’t an error. At the point of job release we ask a number of edge nodes (with a small delay between asking each node) to print the document if they can. Whichever edge node is fastest wins the bid to print the job. 

failed to update job state: job state transition not allowed just means that another (faster) node is printing the job so our state machine says ‘No, you can’t move a job back from printing to released, it’s already happened'.

Troubleshooting using a Job Trace

The first thing to establish in Job Trace is whether the failing jobs have a common edge node that was part of submission or replication. 

The problematic edge node is often the one that submitted or first touched the print job. After you find the offending edge node, set it to passive: 

1. In the PaperCut Hive admin portal, under Edge Mesh, find the edge node.
2. Click on the 3 dots menu and select Demote. 

Setting it to passive isn’t the final fix, however it can help to buy some time while figuring out what may be an issue with the edge node or the network.