Introducing JobQueue for ZendPHP

Job queues are a popular tactic for PHP developers who want to defer, schedule, and manage recurring jobs within their applications. Until recently, however, there wasn't a tool  purpose-built for PHP applications — making it complicated to fully deploy and manage an efficient job queueing solution.

In this blog, we dive in on the basics of job queueing, give an overview of available job queueing tools, how they help (or hinder) PHP teams as they queue and scale their applications — and the new job queueing functionality from Zend that can make queuing a less complex and more efficient process.

Why Is Queueing Necessary for PHP?

What happens when your application is receiving a lot of traffic, and much of that traffic requires data processing (e.g. processing uploaded images or videos, crunching data, aggregating data from multiple web services)? Often, the user submitting the data or information, or making a request for it, does not require it immediately — for example, if a mobile application is submitting information via an API, it may not need confirmation of completion, only that the request was accepted. How can you prevent these sorts of requests from tying up your web servers?

The common answer is to add queueing to your web application. Accept the data, queue it for later processing, and return a response immediately. There are many options for accomplishing this, but previously there were none representing a perfect fit for PHP applications.

Schedule a Custom ZendHQ Demo for Your Team

Want to see how JobQueue works? Want to talk with our experts about how ZendHQ will work in your environment? Schedule a custom demo via the link below.

Schedule Your ZendHQ Demo

Overview of Popular PHP Queueing Technologies

When it comes to queueing, you have a variety of options, but not many that are a perfect fit for PHP use cases. Those options include topic queueing technologies, PHP queueing libraries, and custom-written queueing infrastructure.

Topic Queueing Technologies

"Topic" queues, such as ActiveMQ or RabbitMQ, allow you to send a message to a queue topic. Workers subscribe to topics, and then process the messages as they come in.

However, when it comes to PHP, this would mean writing PHP workers that subscribe to the queue and which receive message notifications. PHP, while increasingly robust, was not really designed for this sort of thing. It operates under the assumption that it gets torn down after each request, allowing it to release memory and resources; long running processes do not benefit from this architecture, which means that sooner or later, your PHP process will end up dying due to lack of resources. As such, you will require some sort of process manager, such as supervisord, in order to manage your workers.

Let's count up what you need: a PHP extension for communicating with the message queue, a custom PHP script for subscribing to the queue and processing jobs (and, most importantly, validating the jobs before processing), and a process manager. This also likely means a separate machine or container for the worker, which means additional deployment tasks.

One other thing to note: with topic queues, every worker that is notified of a message processes it. If you want a message processed exactly once, you will need to write logic into your workers that can add locking mechanisms to prevent duplication of processing.

PHP Queueing Libraries

Another queue option is to use a PHP queue library. Within the PHP ecosystem, there are a number of queueing libraries that allow working with a variety of backends, from relational databases, to key/value stores like Redis, to even topic queues as outlined above. Examples of these include Laravel's queue component, and the Enqueue library. The benefit to these is that they provide scripts and utilities out of the box to enable job locking and worker creation. However, they really only allow for simple job deferment; scheduling jobs or adding recurring jobs requires different libraries, if they are supported at all. Finally, you run into the same problem of requiring a process manager for your workers.

One thing to be aware of with PHP queueing libraries is that the backend you choose can make a huge difference to application performance. Having the ability to re-use your relational database for queueing can seem like a fantastic idea — until you discover that the workers you create are now polling the database repeatedly to discover new jobs, adding load to your database server!

Custom-Written Queueing Infrastructure

Finally, you can also write your own queueing infrastructure, using existing queueing backends, or non-traditional queueing backends such as Redis or Beanstalkd. While this is often attractive from a cost perspective, it also means you will end up writing and maintaining your own scheduler and workers, and require a process manager for your queue workers.

What About Non-PHP Workers?

Using PHP is fantastic...until it's the wrong tool for the job.

In some cases, a big reason to have job deferment and queues is to allow processing data using an entirely different runtime, or a third party service. As an example, you might have written your order fulfillment in Java, but want to use PHP for your web API due to maintainability and ease of deployment. Using a queue system, you could have workers in Java that handle the data that PHP queues!

JobQueue for ZendPHP: Queueing in PHP Made Easy

We have just released JobQueue for ZendPHP. It's goals were to (a) allow for job deferment, scheduling, and recurring jobs, (b) provide the ability to process jobs via HTTP or CLI workers, and thus allow for non-PHP workers seamlessly, and (c) allow for minimal infrastructure to facilitate each of these.

On top of that, it provides monitoring capabilities for queue jobs, and can notify you when a job has been delayed longer than a configured threshold, or when jobs fail, including allowing the capture of worker output to allow you to better debug worker issues.

Ready to Explore JobQueue for ZendPHP?

Schedule a demo, or try ZendHQ for free as part of your 30-day ZendPHP trial, via the links below.

Schedule a Demo  Try for Free

Additional Resources

• Product Page - ZendHQ
• Product Demo - Schedule a ZendHQ Demo
• Product Page - ZendPHP
• Blog - Exploring ZendHQ Role-Based Authentication Features
• Blog - Exploring ZendHQ Database Query Introspection
• Blog - PHP Code Tracing With ZendPHP and ZendHQ
• Blog - Debugging PHP Segmentation Faults
• Blog - Finding and Fixing PHP Errors in Production
• On-Demand Webinar - Observability and Orchestration With ZendHQ