Back to connectors… Back to Magento...

Batch Product Upload (Service Unavailable) (5 replies)

Raoul Duke
7 years ago
Raoul Duke 7 years ago

Hi, we seem to be experiencing issues with the product upload where it seems the batch product upload is failing after sometime.

We are receiving the following error:

The data import incurred an unrecoverable error and is quitting. [Service Unavailable]
The data import incurred an unrecoverable error and is quitting. [Service Unavailable]

After some investigation we've determined 2000 products were updated, but no more.

Support
7 years ago
Support 7 years ago

Raoul, our batch (Magmi) product upload, processes the products in batches of 2000, so it appears the issue is occurring after the first batch has been completed.

Can you explain a bit about your environment?

Raoul Duke
7 years ago
Raoul Duke 7 years ago

Our infrastructure is as follows:

  1. 1 varnish server + nginx for dealing with ssl encryption
  2. 1 load balancer nginx
  3. 2 app servers apache

Whenever a user accesses the site the request is first handled by the varnish server, if there is no cache for the specific request the request ends up in the load balancer which does a round robin to the 2 app servers. 

Support
7 years ago
Support 7 years ago

OK, thanks for this.

We've had a similar issue before whereby the NGinx server was terminating the connection before the request was complete and the reply could be sent back to IMan.

I suspect there's going to be a similar issue.

Raoul Duke
7 years ago
Raoul Duke 7 years ago

OK, thanks for the reply...it really helps.

We did some analysis and in the frist instance we changed the load balancer settings for client_max_body_size and proxy_read_timeout. They were set too low and the request did not get sent to the app server.

This helped the issue, but didn't resolved the problem. We then altered the varnish configuration which was also set to a too low value for the request to be completed:

initial values:

first_byte_timeout = 15s;

.connect_timeout = 30s;

.between_bytes_timeout = 30s;

The problem here was the first_byte_timeout since the app server too almost 2 minutes to return a result for processing the feed.

Voila!

Support
7 years ago
Support 7 years ago

Raoul, thanks for getting back to us with the issue; we'll update our documentation to include a discussion on this.

Contact

Realisable Software Ltd provides code-free, cost-effective applications integration solutions for SMEs. Our core IMan product is designed to integrate almost any application with a number of Sage solutions and online payment processors.

Looking to purchase IMan, please see our resellers here.

Realisable Software
Ph: +44 (0) 208 123 1017

Copyright © Realisable. All rights reserved.
Realisable is a registered trademark

Close

Request Demo

Realisable Software Ltd provides code-free, cost-effective applications integration solutions for SMEs. Our core IMan product is designed to integrate almost any application with a number of Sage solutions and online payment processors.

Looking to purchase IMan, please see our resellers here.

Realisable Software
Ph: +44 (0) 208 123 1017

Copyright © Realisable. All rights reserved.
Realisable is a registered trademark

Close

Access Downloads

Realisable Software Ltd provides code-free, cost-effective applications integration solutions for SMEs. Our core IMan product is designed to integrate almost any application with a number of Sage solutions and online payment processors.

Looking to purchase IMan, please see our resellers here.

Realisable Software
Ph: +44 (0) 208 123 1017

Copyright © Realisable. All rights reserved.
Realisable is a registered trademark

Close