Ecommerce order processing automation
A way to automate order processing in an ecommerce application. Long story short: delegate everything that can be problematic or waste someone else's time. In the end, your job is to sale things, not put them in boxes.
A few weeks ago, I had the opportunity to collaborate on (nearly) full automation of ecommerce order handling. Our objectives were as follows:
- Integration with payment gateways
- Integration with an invoicing system
- Integration with a warehouse/logistics center
- Full automation of the “happy path” – if no unexpected errors occur, the order is processed without manual intervention
- Manual error handling – the administrator receives a notification, and all held orders are manually reviewed.
Long story short: delegate everything that can be problematic or waste someone else's time. In the end, your job is to sale things, not put them in boxes.
We used the Magento platform, so some of its characteristics may come through. Implementing a similar (or even identical) process is also possible with other systems.
It’s worth mentioning that throughout this text, you’ll find phrases like “we used plugin X” or “we utilized integration Y.” Our job wasn’t to reinvent e-commerce or payment gateways. We aimed to leverage existing tools. Occasionally, after implementing an integration, we’d hit a wall and have to look for an alternative. I still believe this approach is better than starting from scratch every time.
New Order Comes In
As mentioned earlier, we used Magento to create the store. A new order is saved with a pending status and sent to the payment system. At this stage, not much happens. Since it’s “pending,” we wait…
Payments
This was perhaps the simplest integration (in terms of implementation) in the entire system. We used the services and plugin of a quick-payment provider, which is the only payment method available in the store. No traditional bank transfers, etc. All responsibility for collecting payment from the customer lies with the payment provider. Our role is to send the order data and receive a status response. The rest is handled by them.
Instead of ready-made solutions, you could use “semi-ready” ones (like Omnipay or Payum) or even build everything from scratch. However, since a solution already exists and fits the requirements, why reinvent the wheel?
Back to how the payment system works... Once payment confirmation is received, we move to the order fulfillment process by triggering two actions:
- Changing the order status to processing
- Issuing a Magento sales document
The Invoice
We decided to integrate with an external invoicing system. To minimize interference with Magento’s core, we used its invoicing system as a trigger for our actions (we call it internal invoice). After receiving payment, Magento automatically issues an internal invoice – and its responsibility ends there. The internal invoice isn’t sent anywhere; it just exists. The issuance of the internal invoice triggers an event that allows us to forward the order to our system. We get a response, save the number and date, and everyone’s happy. We also added a “generate sales document” option in the orders panel – just in case.
The 3 actions – issuance, saving, and document dispatch – operate independently via queues. After the document is issued, we publish a message like “save document X and associate it with order Y.” Once saved, we publish another message indicating the need to send an email to the customer.
We need to slow down a bit...
Document Generation
Since sales document generation is asynchronous. What happens if the document hasn’t yet been issued and someone triggers the action again? We end up with two documents for one order. Accounting starts calling about refunds, corrections, and other issues. To prevent this, the system first checks whether the order already has a document number and issuance date. If it does, the action is halted.
To prevent this, the system first checks whether the order already has a document number and issuance date. If it does, the action is stopped immediately.
Over time, another issue emerged: the document was correctly issued, but we never received a response. The fix was straightforward: we query the system for the sales document associated with order X. If the document exists, we save the response and terminate the issuance process.
The next step is to notify the customer that we have a new document for them. This happens when the response with the document number and date is saved. There’s not much to elaborate on here – once saved, send an email with the PDF. Done.
Error Handling
Error handling for document issuance is generally unique to each project. Fortunately, we were able to allow the fulfillment process to continue even if an error occurred. If we received payment, we must issue the document. If for some reason the document wasn’t issued, a notification is sent to the system.
Shipping
Just like with sales documents, the issuance of a Magento's internal invoice triggers the order shipping process (independently of the sales document). However, the rules differ slightly here – any error results in the order being put on hold for manual review.
- The logistics center didn’t accept the order? Hold!
- Product out of stock? Hold!
- Invalid address? You get the idea.
A critical aspect of the logistics center integration is the principle of “log everything.” Every piece of information received from the warehouse is saved in the system. However, the store staff only sees the data relevant to them. Notifications about held orders and the reasons for them are valuable. There’s no need to inform them of successful data synchronization between the e-commerce platform and the logistics center. Someone referred to this as “minimizing informational noise,” and I completely agree.
For the “happy path,” it’s all about mapping logistics center statuses to e-commerce statuses. When the package is handed over to the courier, the status changes to in shipment. Once delivered to the customer, the order is closed. That’s it!
Summary
I realize that the automation process will differ in every project. Nonetheless, I believe we took a balanced approach and achieved a healthy middle ground between paranoid logging and confirming every step versus ignoring errors completely. Where possible, we retry actions that caused errors – but with a retry limit. There’s no point in endlessly attempting to issue an invoice if the VAT ID is incorrect or data is missing.