Hi all
We are working on a data migration to migrate multiple stores. Each store uses all or a part of the ~150 virtual products. Each virtual has between 10 and 1k variants (combination of 2 to 6 features...). This migration has to be done in less than 5 hours. What do you think is the fastest way to migrate these products and its 300k customers? pure sql-scripts? postgres datafiles? ofbiz-xml-data? Btw: The production installation will not be inhouse - we currently use xmlrpc for the migration, but it is just to slow: I have written the product part and it is running since more than 8 hours. DB, Ofbiz, Migration all on the same dev-machine (16G mem, amd phenom x6, SSD, Postgres on Raid10) - yes, xmlrpc is really to slow for bulk migration.. Regards Greg |
Hi,
Prepare ofbiz xml-data with ofbiz system or ftl or xslt or other etl. Desactivate your eeca on spot server and load your files. Nicolas Le 08/08/2012 10:52, greg jocher a écrit : > Hi all > > We are working on a data migration to migrate multiple stores. > Each store uses all or a part of the ~150 virtual products. > Each virtual has between 10 and 1k variants (combination of 2 to 6 > features...). > This migration has to be done in less than 5 hours. > > What do you think is the fastest way to migrate these products and its > 300k customers? pure sql-scripts? postgres datafiles? ofbiz-xml-data? > > Btw: The production installation will not be inhouse - we currently > use xmlrpc for the migration, but it is just to slow: > I have written the product part and it is running since more than 8 > hours. DB, Ofbiz, Migration all on the same dev-machine (16G mem, amd > phenom x6, SSD, Postgres on Raid10) - yes, xmlrpc is really to slow > for bulk migration.. > > Regards > Greg -- Nicolas MALIN Consultant Tél : 06.17.66.40.06 Site projet : http://www.neogia.org/ ------- Société LibrenBerry Tél : 02.48.02.56.12 Site : http://www.librenberry.net/ |
Hi,
Thanks for your input - i have done some tests now: 1. I have created all products, features and variants through xmlrpc (3M records, 12h..) 2. Exported ofbiz-xml-data 3. pg_dumped 4. imported xml-data on a clean install: ~40m 5. imported pg_dump of the same data: ~7m I think we will choose ofbiz-xml-data, its fast enough! Regards Greg On 08/08/2012 01:45 PM, Nicolas Malin wrote: > Hi, > > Prepare ofbiz xml-data with ofbiz system or ftl or xslt or other etl. > Desactivate your eeca on spot server and load your files. > > Nicolas > > Le 08/08/2012 10:52, greg jocher a écrit : >> Hi all >> >> We are working on a data migration to migrate multiple stores. >> Each store uses all or a part of the ~150 virtual products. >> Each virtual has between 10 and 1k variants (combination of 2 to 6 >> features...). >> This migration has to be done in less than 5 hours. >> >> What do you think is the fastest way to migrate these products and >> its 300k customers? pure sql-scripts? postgres datafiles? >> ofbiz-xml-data? >> >> Btw: The production installation will not be inhouse - we currently >> use xmlrpc for the migration, but it is just to slow: >> I have written the product part and it is running since more than 8 >> hours. DB, Ofbiz, Migration all on the same dev-machine (16G mem, amd >> phenom x6, SSD, Postgres on Raid10) - yes, xmlrpc is really to slow >> for bulk migration.. >> >> Regards >> Greg > > |
Free forum by Nabble | Edit this page |